Mercurial > genshi > genshi-test
changeset 820:1837f39efd6f experimental-inline
Sync (old) experimental inline branch with trunk@1027.
line wrap: on
line diff
--- a/COPYING +++ b/COPYING @@ -1,4 +1,4 @@ -Copyright (C) 2006-2007 Edgewall Software +Copyright (C) 2006-2008 Edgewall Software All rights reserved. Redistribution and use in source and binary forms, with or without
--- a/ChangeLog +++ b/ChangeLog @@ -1,18 +1,189 @@ +Version 0.6 +http://svn.edgewall.org/repos/genshi/tags/0.6.0/ +(???, from branches/stable/0.6.x) + + * Support for Python 2.3 has been dropped. + + +Version 0.5.2 +http://svn.edgewall.org/repos/genshi/tags/0.5.2/ +(???, from branches/stable/0.5.x) + + * Fix problem with I18n filter that would get confused by expressions in + attribute values when inside an `i18n:msg` block (ticket #250). + * Fix problem with the transformation filter dropping events after the + selection (ticket #290). + * `for` loops in template code blocks no longer establish their own locals + scope, meaning you can now access variables assigned in the loop outside + of the loop, just as you can in regular Python code (ticket #259). + * Import statements inside function definitions in template code blocks no + longer result in an UndefinedError when the imported name is accessed + (ticket #276). + + +Version 0.5.1 +http://svn.edgewall.org/repos/genshi/tags/0.5.1/ +(Jul 9 2008, from branches/stable/0.5.x) + + * Fix problem with nested match templates not being applied when buffering + on the outer `py:match` is disabled. Thanks to Erik Bray for reporting the + problem and providing a test case! + * Fix problem in `Translator` filter that would cause the translation of + text nodes to fail if the translation function returned an object that was + not directly a string, but rather something like an instance of the + `LazyProxy` class in Babel (ticket #145). + * Fix problem with match templates incorrectly being applied multiple times. + * Includes from templates loaded via an absolute path now include the correct + file in nested directories as long if no search path has been configured + (ticket #240). + * Unbuffered match templates could result in parts of the matched content + being included in the output if the match template didn't actually consume + it via one or more calls to the `select()` function (ticket #243). + + Version 0.5 http://svn.edgewall.org/repos/genshi/tags/0.5.0/ -(?, from branches/stable/0.5.x) +(Jun 9 2008, from branches/stable/0.5.x) * Added #include directive for text templates (ticket #115). + * Added new markup transformation filter contributed by Alec Thomas. This + provides gorgeous jQuery-inspired stream transformation capabilities based + on XPath expressions. + * When using HTML or XHTML serialization, the `xml:lang` attribute is + automatically translated to the `lang` attribute which HTML user agents + understand. + * Added support for the XPath 2 `matches()` function in XPath expressions, + which allow matching against regular expressions. + * Support for Python code blocks in templates can now be disabled + (ticket #123). + * Includes are now processed when the template is parsed if possible, but + only if the template loader is not set to do automatic reloading. Included + templates are basically inlined into the including template, which can + speed up rendering of that template a bit. + * Added new syntax for text templates, which is more powerful and flexible + with respect to white-space and line breaks. It also supports Python code + blocks. The old syntax is still available and the default for now, but in a + future release the new syntax will become the default, and some time after + that the old syntax will be removed. + * Added support for passing optimization hints to `<py:match>` directives, + which can speed up match templates in many cases, for example when a match + template should only be applied once to a stream, or when it should not be + applied recursively. + * Text templates now default to rendering as plain text; it is no longer + necessary to explicitly specify the "text" method to the `render()` or + `serialize()` method of the generated markup stream. + * XInclude elements in markup templates now support the `parse` attribute; + when set to "xml" (the default), the include is processed as before, but + when set to "text", the included template is parsed as a text template using + the new syntax (ticket #101). + * Python code blocks inside match templates are now executed (ticket #155). + * The template engine plugin no longer adds the `default_doctype` when the + `fragment` parameter is `True`. + * The `striptags` function now also removes HTML/XML-style comments (ticket + #150). + * The `py:replace` directive can now also be used as an element, with an + attribute named `value` (ticket #144). + * The `TextSerializer` class no longer strips all markup in text by default, + so that it is still possible to use the Genshi `escape` function even with + text templates. The old behavior is available via the `strip_markup` option + of the serializer (ticket #146). + * Assigning to a variable named `data` in a Python code block no longer + breaks context lookup. + * The `Stream.render` now accepts an optional `out` parameter that can be + used to pass in a writable file-like object to use for assembling the + output, instead of building a big string and returning it. + * The XHTML serializer now strips `xml:space` attributes as they are only + allowed on very few tags. + * Match templates are now applied in a more controlled fashion: in the order + they are declared in the template source, all match templates up to (and + including) the matching template itself are applied to the matched content, + whereas the match templates declared after the matching template are only + applied to the generated content (ticket #186). + * The `TemplateLoader` class now provides an `_instantiate()` method that can + be overridden by subclasses to implement advanced template instantiation + logic (ticket #204). + * The search path of the `TemplateLoader` class can now contain ''load + functions'' in addition to path strings. A load function is passed the + name of the requested template file, and should return a file-like object + and some metadata. New load functions are supplied for loading from egg + package data, and loading from different loaders depending on the path + prefix of the requested filename (ticket #182). + * Match templates can now be processed without keeping the complete matched + content in memory, which could cause excessive memory use on long pages. + The buffering can be disabled using the new `buffer` optimization hint on + the `<py:match>` directive. + * Improve error reporting when accessing an attribute in a Python expression + raises an `AttributeError` (ticket #191). + * The `Markup` class now supports mappings for right hand of the `%` (modulo) + operator in the same way the Python string classes do, except that the + substituted values are escape. Also, the special constructor which took + positional arguments that would be substituted was removed. Thus the + `Markup` class now supports the same arguments as that of its `unicode` + base class (ticket #211). + * The `Template` class and its subclasses, as well as the interpolation API, + now take an `filepath` parameter instead of `basedir` (ticket #207). + * The `XHTMLSerializer` now has a `drop_xml_decl` option that defaults to + `True`. Setting it to `False` will cause any XML decl in the serialized + stream to be included in the output as it would for XML serialization. + * Add support for a protocol that would allow interoperability of different + Python packages that generate and/or consume markup, based on the special + `__html__()` method (ticket #202). + + +Version 0.4.4 +http://svn.edgewall.org/repos/genshi/tags/0.4.4/ +(Aug 14, 2007, from branches/stable/0.4.x) + + * Fixed augmented assignment to local variables in Python code blocks. + * Fixed handling of nested function and class definitions in Python code + blocks. + * Includes were not raising `TemplateNotFound` exceptions even when no + fallback has been specified. That has been corrected. + * The template loader now raises a `TemplateNotFound` error when a previously + cached template is removed or renamed, where it previously was passing up + an `OSError`. + * The Genshi I18n filter can be configured to only extract messages found in + `gettext` function calls, ignoring any text nodes and attribute values + (ticket #138). + + +Version 0.4.3 +http://svn.edgewall.org/repos/genshi/tags/0.4.3/ +(Jul 17 2007, from branches/stable/0.4.x) + + * The I18n filter no longer extracts or translates literal strings in + attribute values that also contain expressions. + * Added `loader_callback` option to plugin interface, which allows specifying + a callback function that the template loader should invoke whenever a new + template is loaded (ticket #130). Note that the value for this option can + not be specified as a string, only as an actual function object, which means + it is not available for use through configuration files. + * The I18n filter now extracts messages from gettext functions even inside + ignored tags (ticket #132). + * The HTML sanitizer now strips any CSS comments in style attributes, which + could previously be used to hide malicious property values. + * The HTML sanitizer now also removes any HTML comments encountered, as those + may be used to hide malicious payloads targetting a certain "innovative" + browser that goes and interprets the content of specially prepared comments. + * Attribute access in template expressions no longer silently ignores + exceptions other than `AttributeError` raised in the attribute accessor. Version 0.4.2 http://svn.edgewall.org/repos/genshi/tags/0.4.2/ -(?, from branches/stable/0.4.x) +(Jun 20 2007, from branches/stable/0.4.x) * The `doctype` parameter of the markup serializers now also accepts the "name" of the doctype as string, in addition to the `(name, pubid, sysid)` tuple. * The I18n filter was not replacing the original attributes with the translation, but instead adding a second attribute with the same name. + * `TextTemplate` can now handle unicode source (ticket #125). + * A `<?python ?>` processing instruction containing trailing whitespace no + longer causes a syntax error (ticket #127). + * The I18n filter now skips the content of elements that have an `xml:lang` + attribute with a fixed string value. Basically, `xml:lang` can now be used + as a flag to mark specific sections as not needing localization. + * Added plugin for message extraction via Babel (http://babel.edgewall.org/). Version 0.4.1
deleted file mode 100644 --- a/INSTALL.txt +++ /dev/null @@ -1,37 +0,0 @@ -Installing Genshi -================= - -Prerequisites -------------- - - * Python 2.3 or later (2.4 or later is recommended) - * Optional: setuptools 0.6a2 or later - - -Installation ------------- - -Once you've downloaded and unpacked a Genshi source release, enter the -directory where the archive was unpacked, and run: - - $ python setup.py install - -Note that you may need administrator/root privileges for this step, as -this command will by default attempt to install Genshi to the Python -site-packages directory on your system. - -For advanced options, please refer to the easy_install and/or the distutils -documentation: - - http://peak.telecommunity.com/DevCenter/EasyInstall - http://docs.python.org/inst/inst.html - - -Support -------- - -If you encounter any problems with Genshi, please don't hesitate to ask -questions on the Genshi mailing list or IRC channel: - - http://genshi.edgewall.org/wiki/MailingList - http://genshi.edgewall.org/wiki/IrcChannel
--- a/README.txt +++ b/README.txt @@ -1,11 +1,12 @@ About Genshi ============ -Genshi is a Python library that provides an integrated set of components -for parsing, generating, and processing HTML, XML or other textual -content for output generation on the web. The major feature is a -template language, which is heavily inspired by Kid. +Genshi is a Python library that provides an integrated set of +components for parsing, generating, and processing HTML, XML or other +textual content for output generation on the web. The major feature is +a template language, which is heavily inspired by Kid. -For more information please visit the Genshi web site: +For more information please see the documentation in the `doc` +directory, and visit the Genshi web site: <http://genshi.edgewall.org/>
deleted file mode 100644 --- a/UPGRADE.txt +++ /dev/null @@ -1,51 +0,0 @@ -Upgrading Genshi -================ - -Upgrading from Genshi 0.3.x to 0.4.x ------------------------------------- - -The modules ``genshi.filters`` and ``genshi.template`` have been -refactored into packages containing multiple modules. While code using -the regular APIs should continue to work without problems, you should -make sure to remove any leftover traces of the ``template.py`` file on -the installation path. This is not necessary when Genshi was installed -as a Python egg. - -Results of evaluating template expressions are no longer implicitly -called if they are callable. If you have been using that feature, you -will need to add the parenthesis to actually call the function. - -Instances of `genshi.core.Attrs` are now immutable. Filters -manipulating the attributes in a stream may need to be updated. Also, -the `Attrs` class no longer automatically wraps all attribute names -in `QName` objects, so users of the `Attrs` class need to do this -themselves. See the documentation of the `Attrs` class for more -information. - - -Upgrading from Markup ---------------------- - -Prior to version 0.3, the name of the Genshi project was "Markup". The -name change means that you will have to adjust your import statements -and the namespace URI of XML templates, among other things: - - * The package name was changed from "markup" to "genshi". Please - adjust any import statements referring to the old package name. - * The namespace URI for directives in Genshi XML templates has changed - from http://markup.edgewall.org/ to http://genshi.edgewall.org/. - Please update the xmlns:py declaration in your template files - accordingly. - -Furthermore, due to the inclusion of a text-based template language, -the class: - - `markup.template.Template` - -has been renamed to: - - `genshi.template.MarkupTemplate` - -If you've been using the Template class directly, you'll need to -update your code (a simple find/replace should do--the API itself -did not change).
--- a/doc/2000ft.graffle +++ b/doc/2000ft.graffle @@ -201,7 +201,7 @@ {\colortbl;\red255\green255\blue255;} \pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\qc\pardirnatural -\f0\fs22 \cf0 ...}</string> +\f0\fs22 \cf0 Transformer}</string> </dict> </dict> <dict> @@ -1249,7 +1249,7 @@ </dict> </array> <key>ModificationDate</key> - <string>2007-04-13 15:15:25 +0200</string> + <string>2007-06-17 13:12:07 +0200</string> <key>Modifier</key> <string>Christopher Lenz</string> <key>NotesVisible</key>
index d0209916dd84c14e852056736dffa991c1cc1506..b4ce041211763686822d58697d549d3a48137327 GIT binary patch literal 29368 zc$}2F1yEc~)aQM1cY*|m;2M%3!JXj2o#5{79w4~82MF%&?yiFd*TH>|kLP`M_uc(g zYWGx4RoCsQzPIi4?|=G)Dac8nA-_ik008Zqq^KeQK&`(05)q-_o~$G-!UF)tM{^Mo zg>NDv<O&Y9CgxVg0N|F9sOHW!=e;P<nW}#RQSFQIriUa^Ls%#v90NasQU~yIhq8Ws zkak+Z7W<$<`F;7jlqB|WT7psJ-gExx>AuItesk6_pRf1CL)PR&#@XFnYlR~T5T(Hz zj2!{(7c*s7(w<5&(LawCLIRDN<4?v$9!eYZ9rfg-Y*S$Qh@2b%Dsj(orR@4*ku6}@ z-I-ag!RDKzyJ4@X2moNEjIYKCh>`+s&_0{bK-V?|g$EjtSLUK729RI@QepCS3?K{+ za4*Sk#{v}L04u0YI~1Vm;iV@U%8xk?6BFGJ6=upY_%T)(stN_T_VDRrg9g%hf%}JG zEMZ`&28hX~pfC~uqL_df8ihZb05LO=^LOj$CuF`RnZ~b$3wXZrHNZvkBT5B?6JPs& zM?)jRLe0&;;%GG>zyg>Z`}!{oBLLu6MF2e&Ic+u?2P`=PE1=KI(d_T7TZsWa<S9a# z&>Zm>A6D2yxO>}|N}gAa&(MC*0L6K5QK*2_`!9h$LGI#rgp7kR-S>gq!73@`nZ3y< zqdMc3eRon_<`U64Kc_%CXJQ}hMW=<Ui&_^y^nGA17@FZ`m#v|4&_eF<bh`7P8=!kf z$64H_tfOo?6Eiip=rCJ&%2S|Ju~K-a=<D(-FQQAOVd#zHsU%QKcl9~kYgDRKYvIny z!#V^*X4(7e{hH4xmUk#9Y(-;8Y*nROc66<;w|jPxv4>}^2zO(TM@F3%`SA>c98Q;< z4^a~(38gy(FAgEfPQkiIE?6L}Pdt@W_Sfkz&@Yu}s=;J3GMOBCW$wdNC-!Fc0v=~3 zuqXU2t}V@J-3}qBmR$oIM7i(gNZ`mZIl0%-F_o&8nXQAVa^Um5QnHRf<!N)Ky;Dj$ zD%m5!Jn<@tRpnZ{LowB^zg?<6t=wbJV!uA%y+!%F^TqLsbKs3L=gfOA+oR8yUs%?T z%u6ifEH2FF+nL=w^R1iOA0~pM{<KbHrs?psHk?MTot?1w#$3i+4?R|I;hfT1%5U%= zAJ8Am9#Zi&y64VK&UU^yT+grh*DMYP`irRcdam2GX^^@<H@JU()fr0}^jx#wv|sMK z51AOAiAGYoHxDG7)-ZB0sBnC#l)pg9%67(cWwG%hd|GZwcxE;NUAFEZ*RZ!Nw<Wcc z*6G*L*EL_}yB=S<oX8`jF-6|8Zs_6Nz&%5M5^Vf6ug^8vH{g|}5?dZk6(=3k8owB9 zD^6}V$Rh-k+onS%W@OKw2QqMVk(c^GO<ged+uPW{_-e;yNZ>ocMs3MZ*@wKcJcHjJ zl97K%i(`Mg%aw|@y6A<=HJ&=QV?7T)qrh*VaYlB-8jk!Ju$r*yHVUg3*0R>BTxMF9 zBO$2v^f~YFt`&(i1m$`aG3}!!8!g7$44)p7g0lK`zpQ8S_ukvWvIwZ;wfZ`Ajdu-` zcl4@u(~6u1ReV>DQuyPnpG~zx6_{5NJ<)f<zel1)aCgbIjnob+WpFs$@jDWCTDkVK zqKC&oqS76^gq$y5`0&<f($}ISL#K6?#peKDQf`43@?18sf$DVd)|RxfuMT}NeK~8Z z#=7#fLvwk}vy`Wbr!!=}!Ls~|Y&~b?;=E32W!v`|1-mP4!-4Xk!asX4cpIukD|psl z!Jem~PF1c?ZIEWi-^bJKKOy~V#TaxLI;4kuc)mUNMi=I<wds8Oo>Y&-*^Hfky{ea9 zPmFg<gP`sfEds3qXGKVQn{5LmP&bg<o_Rga)`CCIyv=;n1qMzenIer2|A^Pq#~Z5_ zGa5Hl(<t{-u-8q|AGr9*)$Boh%MMZRFX{#w9?#`&ptIJKJ61+cHL8Wr3k!?soRpj@ z9Gx#s4ZEGpEqs@sLQI76p#WBOVW<G|V!DrWI8Mj|ffE_)1jl`t!|L6idwPZ~`dtFr zqqfoZ@fv|>*vik@pF2Z=)~;w0GFSe0W$#AT-?5y0$P2^iu?~w0TU*m%$fDb1H2*+N zAxq0geI?kS$Rfrs*liWPqx;xC`wJt1At7GbO9gFCbgp4u;&a*;$2#G9=DV=7{KK|s z>#5PXyNycQs@@-<s>wggT>X8Mb4xNk8QosfTlK~v^D4*C>8LABF#0Yb7>}TBk~Wfd z40oQ*%rF+A3Zq8JJFaM>NtsIRGiiR6d!mj;q%|M!K8n{czlSd4nFW0rHfcopKjaz_ zl-{ELCB%hT6*#$M=D5ZlbCk<81E(n>Wl9;AZG-k_I2}lReO>V(@k7IN388cO=7Tfa zpX4&6#$>)`deQ7!S*TjL95wG`#P8!cY_s<5i0#CqNCtP{>x7^Dl4UV}E>NJ<9?^ZQ zJyCj45L?_WO*E|xQsGuD)!l7tu`6#atm4>d(Iv55lYHJK%0;t46R>gNpjl(LG_W`_ z=bv?`N-T1|cd;kR)v55_l~a>g&=D?D#dlqtY4?Y?wpfxT@V4FmIk10-e=J&?T)(lp z+04@F2tBIw;pXUiUA8}enwlco#F|+tbd50NKU<*{<^L^fBO@kLIaxR!?t28uxa@cM zW=5zXROzFm_t4!8{+18*l;#Zp*GGVREx?irM2P{W(7^92;JySnQHLU~2dF=w(S7q3 z#GZT^eNuqpLHwyq7s$f~yMd`H`u?METdewTz+4<YRs~l;3S~D;N>n9ZH?Lv3#!StL zrUj=x>ia;DL4{$*O!0L83}4QE&Rt$*L6JVVTCGZeFoW<t0f<n95R#IR;*i>q@|*%u zuhS?|f6$oKXk7YZ<#^I@a>ljFwQkqq5asY-H*TlY#Obc+{@_-6e|vFr6%We>P0e%d zDsv)(oq%<M{|iLjaU5*d78)=_$D*XHb{(!hY~yKPch8LE`^{%~P;q*uW`>HLux7c& z(E;uf?~(|4-gEVq{j5D-QWab2y)u7+K~7DDhX}Z;CE+n)z<3;Bb}u&u^FY|0R-N8m zvw0Xky51%e{C@wF5Gz7QhFAtXo-uxbhdcVA)Zo}9{SJZPL>c38MbF92J9OnR=1MG? zp40eFG+L;VZ~c(}Q^#J=d&+2EuP)j+PX*g5AxTD!R7L4Tl-BnYWbV&@q@0ecpR1#{ z!O_t)urw7Ps0-~1lL{G)S0|mm^7-D1a=bRTT;Vk<xrf2!1?!0UBon-pFgmkHFjz41 zm8h5SR0PkD=w_F7&i!4db34{?)L<-@ZGJW#tewd7+_BjX35ZtBx)vxs=7VG`pFK&v z){kMH5seKU9Xu`d-LGKul7$p)O!%a{)>XX(>wh!TFzz>DHluo$7{4vuxDtUw@6wm_ zS2PP0QWs9a`qqbeA!-oAyMuR$tRa4pGxy!*&-NdwImbB~O=NJygy4o)rG5gNkqn<n zs$4irY(=te41Z_`$!Hi_*ShgiH|8)~*Pnru$)@q0!MVY*ey`x*XzEDg@Whd?lj26{ zKkoF&_Q~zW9OAe+o8p??n^>>RF0U`(&vPF}?js-eVQXAZRz5b?3QlPvVp)L1u#(<o zVbgcJha!h|VDr?j2oqDU(+DLd#CowNGFnr#Gb^!5$80BhejqC+FJfz;wXED+f7fq8 zEpN=SE5UG4Vo?%qp>hO^h85|W)~~6%oVPp+v2f0FLc8I*p}DpN3m6N2{>;`xm>;<> zqnqZEqQl8rKU6zbGg8NI?6}MU5$w<bb!$RgLP!ViLOOUpwT8AHwYoPAdf<AxxR2jg zZP$B0BQ^VAgr!6M^7sFJnRD?&JJj~z-Q^aRtcAmpV35&H^&hoq<d1}d{QaN-&a9=x zn5?E)+>9rRc`C5%&*Ng(qA=5>8o7+^iRLs*2Yq&d0)>|?1^Y|ABWe`m#CJhb7mSYc zoE81^%X6DYi|-$aj?-#1DQ!F((p}A*q&8OO$00G>A6JxaNwfHK@n=}Sz%wCS^x9J; zi=Ra^%Z18(mE9df9S>&?V!maeXUf(StCX<u0r3mxL;1m`h|ui_eu|*<lO#_<gc}z5 z8taZrHUj%DtmRjlAWGiaRPvM(0ksUJNvK@x>E|`dXQ4;DGJ;#wTUwZ4{a@4Z_9??* z<}ske2R>=?QFcpcB780CR&<DE6jjDa24@qxy|umCCD(bLd-{X$eZf5~+=wft>&WR7 z%@RvO!{|44VK<c>tO~*Fs+3cks5P=aB5K7CnFU6_ISM?Dei;utz+duS)*^#FYwqCb zj_5Dwy}$dEC>DDtqE*`DRBOfSI_j3{wHoWCPMEJ5dI&w9LafmuQ0pU=Bkt0M(nr$i zQy4Y>XdP*$YOJidfqoK)*7A5(jdg%>C+(M!GT?MR>twD{u3hmh^U<~udL^8bb{q!b zb#nXW0+22z#XEh(1&nWq-h%qvTcXSu5IlWs_m39T%lw5DNdl*CPd!MLLH_XwagR#c zibb>0nOmb)d?I7mL!?8)lhixrxOVSY@FTT7)Mu1cb-ru6S1{aIILZEX4)KVD&q{Y} zOiMG=RI!n7C~}otw?1ZXU0kug`TIh3Gx$y(lofn~dRr_o>0W(5bFFgG)hHC7i_)t^ zHDbFhK^9fr)<`s&c98k|_0OkwUv;<~eO+8gA>XWou4+Mf+xT~t>(c8Rt4SmsM3sLk zE-P&+#>+5l>TK|yLzind2CP}1%5q44PXj(+X#nUl5=Ey`vR@D%cr|W@ylP6?^=-UA z0lI{d8|~iY5O8o?l(r1xM**PX^vVzeekcHl@W|5hFK5ynZK@^!AOLw1NR5uL02I*2 zK{GW?2rL1ZujFk7YqrFs@2xD+053o?k8FBrhm-S_4KzXur~@$!1_!yF_3wKrNnR%0 z4?;K_-lPj)#6~kUbnE)USp`qU22YIXhuSR&&k*w$3DXbR4H1qw0tOvX5Dq&?i0dx} zbQCmtfIqARIke<o1W`oDc(mUSa$#77S6LOME|_=Feu&7O@|l0B6#TCTUtKfrp+M;g zl4$1V(POj{mOO9_x=I;JtI(@wR;2>JL=m5r4wIA4HheaOuX@OSPG=uA34qA_k)d8o zG2qS-coitHLowk)^xyx%+>(`g3PI>}u>}ARbU#W~z)!9;;`J&!2q*d{3_uV|Kdr1N z_=9W0PZ)`2D`JQuA=Hjih13*FYKbaZNmW@H2Dm4)9&&L$eO6@$0CM1)sF1Q-#>twu zy7FQ>>-kw0=PlP*R~rTDfgd6zk_d9zl=?S8OX^7fbTfyNsh`)P5yM#Ajnn!2QrSn% z^$ruV2}R>rvv|un*7<z$Z01!BQgbx@s*)twefZF#LdFE}rZAtz)=}@Gq53$<k;j0} zNr92EWy|t*lLgy_G7V4b!pcgFAWUJez@e3Q<UyDYIB>?oh!+&N2}l%uAqomKL6~I1 zLaA?mK>XiTg!ZkxJ4a$k9yO^2fn6F`z~B~4c9Jkr%0#IG)ly^j)W+j2W88%4xjAJm zt(BFPWAHCb_O-D7t)(UP{OJSmFE^Y|<98iq78WJN#TPEjh!mXt{r$50O}yp%2|6rl z*4E{XmO_ZlVsC}f-AfndHWN(*<MeUyTD+k9wDon}^zl8grrUTO`iM*G+KLK!&}zTa zY5MF}x>-#WVXxVL%Q(WJmAN@Gh!p%m0qhh(_n0kiJ6Wn_>YHGO`?|Xg-&g*qJXMnQ z=X!=zDLOg~(=<JePpCijPa*0S^PcW8s?<vGvU-@pp&0)|Bgi8`m|G%r|Fr&ps^H}~ zlB!9WqqO-eUr`vUUnBQgAJeRJWG*d>*ULsI#>sinkU8>$fTlDEMlG(}2zTw~R$2So z0`o&chE1y9)g|GC2sdiHCY#*^Gh8uhylYc_VUT`GM9*yCAyW}D;m=`)V*gs2aQ$Bv zO|8bN-j-T>nGJ|I4tTSYO99660?*g^STE}pLpvfu?pEh~<5gP=S!)r^8j;EG)6>US zWV@1T1{U@P24>u-C!y%HTuTZB+-Hkt*a7Wm;Cg9#s&}J4l=M?WLaLTldU`t1=v|Nr zZF!}A)Y-QOY?obm1|<d#Q#@6eKlGZN&6Klk&x9X6fP~$oFE_L}FPdp6e#e_o9fSv) zXQbBY`V2gatIMmat7hx@%bC{>hid_x&9(*tT-q!Z`MsOqG#P%}guytka{)ylrpChX z_w<j@v;r|HyOi&?Z12&S0CGOKPOn#Dpvz?^8Nw^!&Otf7PIW~}&P6Fz$w`1Zuxq&p zjkd<y^hm{hf!fO^<F(*#f#TO9qV>t4!Gujl=w^32m^-WbhwiX>x00&AX+(;q+iz08 z+EWE_aa~>bcufU9QZ)9Yx#;K9rQ2VteR(2ch0eyF3@F<PS+t6ZRac>=qGPED%MPN; zRn742-9Oj{%Evc?YgD0JIcQwXELA5Kcz=H58@H$8*o#TZj%Z|$6Qkv;wmQU#W|7ov z@p`!29mS@c0|0kXubG;nERUn&q9j&tN{&7<ObHe@1JF!A(nU0jr-!>&s5T$ZX_&D* z-}CjXJYPK%y-fpwpL(gv{moJ(=dy<>Et778(?vj#L^6wB$5Zic_*M_^Zg3xOi`9eD zXfmr#Lc(?5jB)t3aGUFCM=-~S_l&u0m5`#cPB&<{MJsqp_^d5Y8(9m9y#ZPKL-R8Y zhplVoqJ>1`Q_#dN>HAzjfVrfUzjzbIz2o?-(;H24X1IB)*?cj4+?#b^o@697`#$}h zH<6pAHK?P8Z&&42`f2lIH9QgQ9Tv4cDjdJAa}xUW>}W2*&H@Vm5_x7W<LI1B%;;vR z0XiMy+qLzjbtxk<IyqEoBwo?2Hkxm_=1U|ZRe}LF0<R-vi%aS@p7>~4>UXzYM`jvr zC49stCEas-ylUS2|81>!x~(4{;-y1qJi8qjYfw!(EgfR2^zhWZg1V86B;=gB+z3;N z_SA~to0o}Hq!CuAnn4Q%dPPuH(xRpC3&+&L$+=o#!va($!oqEumUA!^M?~4ipvjkv zr^B~Qgbpk$9gm%)GA}kk$sJE|^M?s#l3zto6DjBfX@shvSh|_9!sZ52s;<<|?MA5J z;ef_;m1A{4=QBUCppbi+UDtB*g303*@9&w3<(Zma$GsoWJx`h=Qy@i2%;Efx)D2}2 z@tv1gX$Ad_^x#D*K#<!})_9vS7TegRmiuw+AujK-b{7&;gAZ&Na)L{it9XUX%;6Xo zO!jx%>$b`V7MjIh@F9#c#jIYBgRW2$uo?GHkH11me@to-B{2^Ec{h-n-@b<2<<m>! zw%)3kBb3;5Ngag8`Fwv7jpAYpGKyNw*88X^&hU4<2b>i>p2~Q~+0hXPcWNc0UR`=r zzKbN4ySo({N%(2%a(fweTdd%FDB$G%@?9*D%Iu<%6b(~|DJi!$Gybo%BQqKH-4Jp8 z$~5>A0~HqvQ<Go7u>w@#yYmeYmRB3ak4&n-^yT{Mkvml?=2x3PPqA3?0caT;Waw}T zW|sC8-E(oGV4BVWf}|6DODvT>Fv~A|v&)}9n!mbxJ8TAjkrdV$hvR#i(878%pSNye zGg7^px0;W#u9piaHxylG`jxa(;Z4o@@fCEgVGgc)N&cC{_yV3-?eLwCpXBefzpKz1 z2*K{1x!rTj_Ho#H4s-_{?rk|TPYIV^#0%{S4>dgZ*QCxscvNhQdCmh>T4i?=S2$=3 zc3G{SE*z+U6wid8{VTCA$1u_i42mzuz2_~^KQ_BQ&a`&+f+gwsuXTH-XwMc>_yF=B z36UT!8&m$5r+R<vF8Ah&TpYi6R~X9cU+*juwR4qu>*zr7Wu^H#g{jO?s}14vXx0Fi zCw_ntwph&JG<nyW$0;iZKILa*;C2B$l|vIAv4OSINs#z=5|rJNXJCMapc#+e;sVNf z0Fhtq_|fI^FoXbk#4rqjmtwa2?$v!V0UBFvS7l63$v-mEUTTn*^K4Da9xN43ow5&= zm7R+;&uH5rF1P+sA)qLwVO#KlXeH@5a}dP`Q55xAWz>OJfNs)EAU(e2Qp4SRYL_U5 z44KhHpqBlewsR7L_b~UMoyzsciARyE#gi4EGWQ3=pXbk(`$c@UbvGx%q=FbZx+)pm zzOoz(q%uDWd=Yp9S>D-|=T2jnf%n|c6=nbEOKJtU0wSH{ymp2^Wko+lBny@!`>icN z36gTC(L!lk%Q)px_&GP|mOM#F@8s=hAlq4!_*2jToJ@Ftfp2C_>uyi5Wjp0-6{t3W z{rglK!RzDuaw&Mkg<CmmBLJ9~K^H7Qg8kd3uLLWGmc)n>hMoi!(4J{<38W1X5-AGI zlXOlMo1jLBJ}die#LxogF?~!cHUZ|4T5);gax$P(;fJY*nSz;{nVFm0Jxc#xcm7W! zHbKI6cjlP@f}gSY3lgbg%^<Naih`riem<M7jXyGf=@ZbH6`4zmTMdnHGQ?jdv!V9> z!EV+WWH(d?*dlcpBk~Kuj%xbS4@t@}ekt<;pmZDEONZ|vxNQCi21&;0e&AGj;9QBK zM;=)+v1`IfN#UOlaC37rt9D;{hv#ZOrYYbT44ZE`)}53cWV9@ZCpgvn4F!+MV+#w$ zui<eoRfq|YFpb{atQ=Pa>>4z3QGP%*mbat$$$VNg@}Tpm5Lz|2KEno@G_}4OFAEa> z-CO1{J`&{q+fcs(Wmbm_kXWlu>;dG{<9cZ|RvKGOfN)6ttX9#fXUd@$QRvnFq+Xlb zh3;g9dbNS#=csW5s8Ur%>VSBh4^p>DtO5ZA1&JZNLph{wOXk&f*sA5K{8Pyv{y`@f zg^tw{UGp)ayR{MwDdI~}%{*+IZsxd9plPM!zOLrhLfvbGsumYm)#22>1XSm4qVr$b zSag2l+50jIOfRE^I)9UnbE<i<($*f`cvW00h}63MGwr=SLndGUfkhrY&Q4{Qw&HdK zpGgaw$<f>A#4J#RG`-F&Dk}Id_6K$(yX~35q9YrP8tixPzSyp}A=x5kzd7g`1mnt- z5+P7)+q;DRmL*&2aV-1f5qa%|QrvbO7YW-}B?sR`+_rYdrrm)ThD1cWE1k^15Hu0I zx*wawSZ6J&cN(=kbcaOIf2dDUXa(ASbW+bdp^tnnk{NO;nlXQKbKByq9z5ViQ3;76 z@4it18y+KAmPYMz6d)h~H7m8YB59JvD-pGj_YD(eT-!4jS@lk<IO^mi&8@S&lnLK) zYwgndw+>z!dy6oR({2R3G39LL^GI;58cjvh`b0xZtu>C?x{*jJE8_nxr6ZOrk)rpq ztylEmik3?av6-EPd!Bj=AzmaMw%x(~&uANCPLd#&XYTzNbYFJ3%2{-l3Zqc0yj}rK zvBOg6xRJ3Sq*1OK-fv*L7=Y=3QmtO5TH3CIQknYpoF=}1V8EjlFDOQ<Z(v}=s3vw+ z_WSJf7FPZ=VopOtL!&EhBR*A71Z}lCF%i-6VdCsUU5A@d%;;_n$m$K@!7iSyEIa(q z*F^%};QGZj9TbObQOa1U0>#q#S@(z&A)!<mEL0I&6ZUs9$K3z)_sjo-T>1Z3DD^q5 z<7LWFU-Q!_!CKRZ6~h1b?4Ej>zKSDkCU9Zj&lVw^t57=s$z+T`_OTK(p|!QunnO!X z4Iiw@LuwXpyY)_1R@P}hGYsp^aBx0tc?(I@v%1;Z+Un@&h=}wO%tx-`tfB9BOw-)m z<rWtg7Zwhz)TfB|^z@Lr@5FA-Mp(!-7Z(1cyzDywgUgGH%`Gf0UBrgw_8ponEG^r0 z!Vh;x5*e+nt$T|Ix#=3~>vftPj3Y`!#S0o57Iq!nTk(v3)q<>W*eoejgZughf{J$y z(@J0eRRA*}*GUy(Tr3hCoLp=e1!`vB$HSszu?Xu2j4p#})a)NS+`Cw{1OiymEZ%gX z)(?Z#M~qRiGbM3Jw8tMRCZkE5C*;3<rqdiJdj^kAq)$R5n7l737M#KP`S_%yk{~vN z3f-BN70X(X$wbB>_*YEvm`$U+AidDEy<?!Ub*8MEm0@zRc4h3FLw-KhQRm;!H6zL3 zWZ_scMn+^YU7WY%p9OJ85%G5q3Ay(%4McXT_CilAl;)dFq|36V6XgJ_E~iQPM*f<3 zm?sNz8=kSJBQ}jn3N+gmPm7wNrh~No@WX(a!InsciVLhbO$sH95q6`NS*yBtVTUX! zUNSC^Qq{_4<@XW0{D-YCjr{DN6?RU|pCpWRENz`;7JHwkr>FC0;H3WA@V(w0!KcfV zCPBUVK^%qPejjF+24TtottR^$gWcweKC$Qnz%TM!L<6VP)8Es&2JSwgZ^Jac*GoFs zkuw+v=Dd)*meNej(rr9`?2_14os2ao-{6g#v}j9l9LLke&~bhTh1+EY!c-asXhG}? zw~=0{X2~Y~mz{0K%$kGWVw((`TeL;*zkcM=w~vs!G`<<s3!Q_JCHwOo_`3{l$1L5# z^I=xb_X|Hhy5U`>Kta8z$3Ltvv&T+-GO~!&WM>m&$V#pHunsiour$ihpk!rZx;*?D z!pw^q&M{oG2^CeCOsj-n^+Sw1=k9hcU;UpD`EnTmv^`sSt%NSAy8Wj5xp$<3Jr>$P zvf1>wFvrQTMGQ=u!!<UXtf~m+xMhe;jIKzOVX$#_`5mVv2h+AJ$Z>#OAlg(l5R}Ej z8olUsw}q9eItcCefn1OUJ4}@Qh6=^7U*XUhIaOwT1*J=X>y4|gj_q6O^x>&TrPg?r z-@^e#?fGgq2b}SdUkgNGk%%yJVgQ@;e*P52vITQorcy*i<QQSfSV`PXalRan>-6x5 zufd|QghV+?2vzK1i+Ue~X=n#QCDJy)Dn1!0AJONHH_3Q7Rvj&;=Lj7Ghk$SdZ49ue zkvyMIn&1Om;}QaY?%Uo0tL+NrxRiiSw(oU!$gXeeCOq)w+t3qWY_wW4Z?>(8UBNw| z6320L0Q3HoF<b8xdg}H;;@$kB6BN_mh6a%G`+7t>M?K=_^8-D8+pC>3GAyDGP#KGI zKIfYVWO{FhaPw_g${b<T8T_tbCbnO?<=4mV5ZcLlD8KAZ?Sk*nn9I!Qf)z*!z$v<A zk(J~u!Y{xyL<R5A{4^<^%u8MOA(#kgSw7hszNf@u|5hAOPnlEN3-4Yf2M-)IM;9_{ z;f_Jut$f{qVh1?WDqH$AGg0kUI-X|W%2na~j{p8AnI1S_+5iW^n&p^K=;`HZA4Mt@ z=mPU`;>jXa0lRuFoto75aDPt^?0^zMI9_^~FH%7H4H^r%!ZC*5@j)7&7R+DGwS5<$ z-NtRkBRA7#vsUdo?2A8g*K=_509?)r93UJmN<;@0gkB0019xQf{LxWzcJez^RpYX$ zgc(rh^^!^=m@EPqO+-cM&bM_ZINEkd3UxgH(<}2Y$N*SPYxhx<kbH^dxuE6S@2Mjg zldkdlp}DKgFwoRd0?=Xw_LCtR@__I0-(&z>v=9LICGAXDyLye^Y3f*i+op4)bPVRY zjX$ZW^7P{Iex(&WsN(MP_eTaeD^@%pot&*UPS=kDxO7n3s-=dKFHaHNlwE7vPEett zZ*bnSY^~e*Yrr(HhCzWIT6;|gU>p4d(^dZ&Vu=GdX*U9hnFTOY#@1vr*OwvC#(qDn z_vb%S3N}q|LSwQEXu~&sEvp!Qc<T%IgC&fypVm7(7Mm&<A?%xz6vWs*Hi^L=XD}Jq z#{!rjvTpaIQGm+HABQ)C(ExNf(zy<>{w2Hs?Uo;3-T+V9z&Zlf4z$8XMd0`MAc*5J zmKzFKX2zp4VE~$T^=s$b)TyAfNvzQ)F_g2WGnxcmIno_XFvzh8$z-RSjHFay{c2s@ zN~_@?EEH}RVSfQb2Kdx)txth->^Y?`is=Vfz?A3!D;r(FJo_69y7+=u{R{8f5WX^H zA_Bm0n^^c<X3dXGXh8|fkWLOO_72G)8Mz9Z5F)AXe%<8d{Eo%URuxfbS)UL9SMaLd zq&AW+CQMF#IZO;>RLpO7c^X9}huE*S7uotVN*A*r7I7J{YOUv`sbPzul><FxU*5h* z_!s^IRILmYOA#PRo;(RmqWFpSO|ZkRV?IGSsV^HZOX6X2m6G@g9}aisFV1T|M0fQU z0)&hqC^2XeGi_rMfi>Fsw{tRiQ*iWv>tb`Kw^u)$`=~SQICk~Xk1zl@sBvlfTnkCp zB_^PNS=E-nC*HSUL7<)Zr+Oj2V03xCI4m`R6`LYHzVx;)B-x1l`nF}tvh?W+-@#Xv z!GQ;y$pYM8j%9ZeOM2`bAiyB#a9wu80R}o{(7;`Pw%mV(M^-?a`)c1m9Av@IyPRY2 zF-EUbz;}u%KFrfor@b8w<{|s}Z(!~7YyI6m>#P{<_$*gNwt!<pR;1+pqX>L#={UaX zNK~7Spt138=ptMcS@oD6%dxZ5SNMI*J_lcJ?aZj}+UGP^YPESRt=<WX1H(?!NLQbQ zi*~QlSrtdC^XGNlQ}goj_QuoqBjacpmmyoFUwvPm`or<x0K!vokV*G5=bHmKnl!D9 zl6a|!2{I1*Nr8bVt3pfj<DyiqNLotjJ&k{KR06_H%U2%T$l~^>i3!gS9hs%Gw}}-h zNsq^KU;j9F*Y3dpL4|3K{O8@@hMeYB+@CLZ(Je)expvQ5G`(v^@UTj5aKd0EBzg-_ zI5rNz9i$1gag+b79Fbc_{E4WmLY~`FG}Fq*RaHye1l!H@x1l$BJYNf!K%_8wpZ6ai zY7|k#s#>Q0v4K$T<5}lxM`Nkr&~r`&U)#r%WkZ1?nbEk@V0{8xHUAihgbqY^puuwX z8t&iVtI;pau!C`#LTF8F37W=|l9ErR6M$XonwXdvfXAdBlag`(_P}z;rlX4=w(bsq zQ!G|yNUULkRK7k|dV71LhXk4k4DX0-+z;(E7?0xp)Ng5R&6G2Bb#=A1J$oD5b{%@* zQ!+SX_U7j+f!C`kkWsAU_2nq5XElEiFZtX19nwFU%E`!#{mp*EirCm#-7*XcjL-n~ z)N2l_r5pu1;2r|8U8?F{nWxzCf1}X&^%9j*SRtV6>gj4y;0%0(c`4|{PsdYQo1)c^ z?{Ki1-g8n+3ho%u3oAICJM?H#J2KijJ9`D&`f7c^=DV3wk}8;4nNJnBka8gSNzdzD z+nK>K^n;-A_fMjiuA_j6fLXorWc&HxF|eY1w8F&sDTz4jXz{jQwU6Q8ejxi4>TGw3 zbYwC-@q{8YpX#+AS@c!~iJaV5!B$r}_R5aOk`=Je&AeK>PON$1{#U=%jjY?%3^d_c zR?cLogU#SrjOSls^oF(XS-)^_rZ=YhF&jsgk4HupTwvT{d7ZI2aJzq-f!l{M_Sv8j zQh=|!!lsOjg^4(#kOxKKx{1#e2TL45aUG)@8|~~E_kTi-S93b(iembypAr$^K?~S9 zOVn%;{?u1={B^|LrP26>ii&kIUS#5e#`Jl!yl`C5XDgKKrmyG&Ev@e@$W)&9BB+s( z{rUHA!#khaBHz^Llhy}V*EY7nPdVdD9$l1fk0$>Hd(pZaWBU%{+U_F)3MbOq=E|Mr zfIk9tUy6Ri|3$Z6(^PNzlJzF=w*~9r%9aVtKNIAcn8#uho6HeKh>L8;2Dr(eG4oDr zCXr^QsX5(uq02vn9CLG7y6k}IPB0L8N~}frIhr1OGL$D5&&v@?;Bsw_Yea!>tjTtQ zYt~{Bo!>AMRQ}K{we%WVt}jsCaBA&2bvv3%;OW*}f%k5j%^fl3c-+@DRU248Kz!52 zA+D8LH;!fI)_jen`d6c!CH`RL$EUg`PSy*r<43~dy$XR3Ijs$4Crw@|rR~jJjvH}# z@|4*g?Ox$XL>PsG(~W<J-=q}p^^^!Uk`C&Z+<Q__j{#}V8EFh}1E}*09@_%jjd-MM zK>Nm*br@V>u@5=fr*x^H=Wyh;WoBGiOVWICabfZU=v`absOVe4MYgY{-+Ra^?|acm z0h_v~+?skKzj_DLQ+vbaUj20eN7=2>`BlMGn1U5K{4sgR&bT<whRwnri-vCw=|KfE zBM(vB+Op=mPEVc3$>)jsh9p~CN#BP%;`?7~4}&lZU+)*5KV9KuL;8OZ7r&e}H!pUs ztS_1hG+OVck)@D)tnAb=fkuDvY_UtG99a8+I0EG5?|bqRzuLuUe?7O`7r0eR*?Q)G z-fkV7(30&6o2-Q4V*`U?-kXR1l7`|a*Q!_?{9P_zy?(Iwp1)jBN)S?X*5TTNoO3NN z3w<I9N`8&dHFsmN#dpc7{x+A}t*JCbe8W~x0V7H|O)~2hq)%Y-Xw|2G7cQyS=wiBw z1a;%-x?P|W?M|vj(`LZmycd2s6R;-M3WWzf1#{wV*?TF;$B3q|NAr_rHwoOS+@{V7 zl@shjF3n61H)SYxA>U#81JpGCT1{f?+u;Tww(UKnViVJbK*%=uRJQ>sc`x*?sD#($ zn36>aF@T8Ec;Mhh-o%USoeLDe?px#Yk#V~6W%#4=Q-upT18v)kmqTV@+ix@1@ANra zP|jv4c=V1v^RCPWZL$H4Q6%()sf>K8olRf6yVkwrP|647;R!KotbH#UO{Ji^PEx72 z{1eFW4qmMig^>k!Y2%0eV#E-_?>ciXB1(B3r4;dgP!nW_9nqIWuckNZWgr=V{V#@P zlr#=o5q!RzXl7%R{ta`M4Y`sXPM@@|sbtlv#%eP3ND|qr;IpjW+touI&@9FUe)|NP zD+bhOO-yXun4~HKnvtET>_d1YtG?&KW|P~z7=RB=5G?)2seEc)3{guy9bEzs5##;M zr>UK~K<k<q<4F_Xwwu=NB}*)#^ld{yS@ss-N&jm~N=l=D!|A4f_WS}(mT5O%YUvwW z)~vvMZY1IU+#_Xr;P(@RlBM}3q*LmdAZwPFo(ZO>hwrZ($&D-=E`M}(aMxOmr)63; zGOJ0s$t%|`kPHrRHtv?`5}bTRscIzc+vuuj^Ibdt<ml1gf`)L^b)~B8HGL9PdzyUA zz?-`yuT|ZAj|+?*-nmj5o1cL7^h^(gzi&4FO-*D}V>x8fQF`2Ep1cLRV!{lIm4lzQ zl$ibO9(`|nu7#~$Q44BqI2*E=R;wOUfA}0^)|Rsrv3X%_8)1)i2lr&~3taDQ6!L*h z^)6w6-4RZPSKNfY9T>sK3d7WoW%=-IH@<Tv@KG_)ns`<_SBd?!al(AHWQT+AvyH!_ z{Q1r4cngD>6E1Ay(A~JgNj*gjPthCu=swUDM_b#NU$+%Z8zC5pd>$`FIdl0#v*!dp zrh{WPAw4DY{7yOLC-InLOkcaRmFGbmevQ^^n{~MPyf(Mr*jU0#Rw?u??Wc68AJU-a zApfS*BIqZ8+~J0(pEf|4k6zK`i^+iCAJUU)pJj?(M;w68|9Ln!YtA@Xl@U@qU4&Qp zh1&8GVyPLMplk5TMvuI=Q(z04T>sv8Zz%qhrJ|t$T3Fy1Ir)bSHuY9B@K9*==Ph>s z@c)3FFmV(vR;#^}+jja&&BU{pt%lkg@sjoN`}glxd*ejM3=toH5<WbC;*<TA#QiT6 zNu=KUv^30`Z00PKQe$$-DKLZ_Is#oFs~E<_XTC|*l$Oov8rLnja1nOxvppufqM-Rw ziAUosp3GRE$IR_xxNzCLg^i@_w-&mN=A{tj<YMSFkYJ0H4lrP-@GcTt8vYAXb$=kB zj}DqYmT;>MiKCl;jMe2&dPXM17Cw#oE2S^-UvmLeFq~w=9B>V33LcZr3OO0Zbb4ZV zPn{Ed`q06f7Jr&31oIY-L(1t_^kylxYrf-u<R}zWpb3)U8*z>d0ey@J9+5!T`<04| zqC-!IBkUC;YWq3L{HK)R&hOCSc*C1FJd(lxZ=o;#QS(}LCXVHxAQ@KNgfP~vBlH4x z2U!J$81eqTePUaO02dYNSr|BD2YIy$ZF*K#*VZ*0T?eI?Czag(M3;`}M$XeOV6Drs z%BJF)QVVD2n{d_+bw$N3@Gnd<F`?Ae**BJXol}xG|D4v=*4EO}62_R)>Oy?~lU|eF zU(|feeS7EjjEHq(RpsU=w_Jl;>x$PiW21wFQNj0jl`q*q1W6GybQj5TT4G{y@(7!Z z#d<Ck0<Tv6DHksmj}MxKwe|Dl>(k`<`T5Syj_bPnzFOzwyl9df4HojqP8P50MU(yZ zz@*P@ZBS6q!_|K0&4LyQ3CXse-AO}ap>x=KnlF|T`okBawT;&5q@IpyD!!J+haH*L zyDGMWD6Uq6N4$KT%y+GOw;lYDhSUek@Nu9lv7o?|YTd?`pP<uIWqEQj!?InugHPbu zcQN9XWiNpI>E*7|_i~ulbUf{*m+YmgxVX5qbQJ6&T`&`iK{Jc}294}l`y}WRIy!5G zvgrW<0m8z<1I>bO8+fuow}b(*Z_CC1dK1@YN0UFq?xd!|=%UhI&r1>4>ByrxY&gCb zpLVq?2G=Tm*IipL8~v{J)<~iR?aDy0>f)6tdRl^IT++YY^QjRwL8?GQLxWZ>!>(By zeQ<nyg8yL$=cj(3!37tX<ZD`n@8fKC6yzbk(&i-alB<2V(64eXLI47q6d$vDYN<2L z--#A@xvy|0IvWpYZh;}xptdsV`BduYqwUmISx;=-2?=Oz-rpEU>pESEQknOnn@jLI z*VY<ZOy=vvTIiN>hXOWWlqAUkM4UP?%eOsfWmT5_k8Mm1lKDPebaZsAudmxLb2)wH z?z~;L)H~E3I2`AIi#XIWDXTmndafWga(g3B3w`}u*W>eYTkqq8>3ra~VzDSC5(dU= zvhPFknb$s#zrX*sW`!b=wra}<0lo_XwSjN2raFm)S?8XE6(xZy)rIdr%!_Fp?wTj1 zFm@$zHds%g$ih_$L7wf)4~G^ibXr}&%dDMGm-wCc{p;<Xt#8)S;^kYJ$MbzLi|w<w z&}rB<SCTOdG_<ZKxc=zqXnA>AP1oJHjfx#OP$G&3gW5+v7!!yyWM?<{PF6SKBp^<S zQ3y2WwDe;9pn&3Csit8@N@PckZ;Ik+*w@gRo2g$S>|=|#PJNsk`zOU9ZKC^HQDA<` zfyPSH_4WiL@Hib^q1XAc8A8@^-h%>|Ok{9>503w4647}2!LT2{;k3<Rcf>T?`*z&c zM~Mk<;|tWM3L<4>ZD~e?V(m{x+4HTZB_4w=Wpd%chVvc;uyRpE^YF%o*7r$=r>A@F z{N&_6C`+A+D(jqS2^nYgkLB2Pd;Y`PRNF+&*;YFO$8=m(+6{RY(p#Hrr1Vw;O$H(e z*DIhebl!&JKko=WW_74n8}u)kaP?2gFB!$<{Wo#OOPCCiiTJ2DDLEvXSK-ZSxKeNa z81R1)mv>72((JokZM!)qI4*oj^yYhB+w377p6@r-Vi8m6{(XI)L?FAGlEcdSOYr<E z?4nTB9EpGqtBhT7kc@}+_apG#o+R@h(nGr3)42-|xj!sz+kfwcR|1s>H8OjCtGTY1 zOJ<*J^YeJ?BQz*~*@H!d2F|B~<nKu%7SUP11?1btPc~pt@PB*oGT(NP1e^L;NWJz7 zU2P7kbVXos-rz^kur_};_T?+B`$$ri?LLt30<I}(gIl)wGl!u`-FjSs0Wl)Gn~r67 z>oWRG7#U0YG1el9$idM4)J;EXU*xmlF;qAJFc<w&B>WjRx%9p2OXc6t*?(0HYq);O zTw7i*f{R-m+3o7jtWw5<&X3;fu^tt#EZ16BihS9lgXtMJAxWtmFzI?M#TPcxnCypt z(8Tk*G2iLK);Z=yz~@hQajeg52R)V-U+_eIhBEtFL&bj+0(k*vJNQ5Kk!dHq$(O*7 z43yA_QK2($Wtb>jBX{`K|Dy=Ya1Ob2!95-NWlBI}cuX-GqiKI5;jqOga#jX2pQHt8 z0yj{$4XFjz%wg;`Ub>zvRT~aQm20`;W;#FzawbK$-ET-cw3aSE@I#tIGcA9O6KkS} z&{U3cxfHj1NkjHBTkrJj{zJT2NxE2Csyv-K9}3*@EG}zyv{O9qZa7xq`WU{%HgSU6 zdfQ0N9eov8yEHX49}&j07H&%nluqOH9AgP>6K$b5ULcNF?Q3{JGEz_DzNZKFp<|$H z@KfE1ND(?;d{XD0L}h#31MP}`B*N9+-A1-wWB#j4V-ZT0G*8WWT&68&<)m!7XU_WY z)j~~;HwEZ*)@r@ebFC!XP}cIeSqCj$??nY|Qp<IJAHIy)wN7n(wfQJGuUqJy8ZRf< zBdfWx@@%z%&=T9X7vpf-74PF_xpRCayfF~UCO#1z5fS0V^>v1Tu6vwl^cY+BW488{ zn7euRY2F%ul6#sTD7n$e|3%T-&C2pY9KinH(7u1`Qx@R`)Z)wiDZ&?30K#v58vL&H z7w5p?Jkn&lNlz{-*EcsaHW{p3-KJzr3Z+pUkBlGM9nVWHWm38YYf8=L)w+4;XzUb5 zat9>Ok&wX`WTm!^B44k&CHGP!u8Rlf9E3tXB1I~+@AmXvh0dclk12IG^b165e?4Oq z+kC41RcN0Ehne3UGr;fVCxXWJoYStT#o)vWATe7AsJHfmYroFlXQPequ@!jR-i*wb zs#WSfZyoL0$5C)5*?!Him0I_Tiud*g{fhL%Ik2%Dknx=y9WMD2*nG|l+^1q<|L*IO zXcR>T90UlBVuG2L^|X1V8Bm+wPFoc4-|C<k1m{Ll&ikFe?N+w8-799f^Y_E#Fv1OB z)|XoudLV^wEfqmAKppt&$xF>Gdr`}m>v!hR_4&LtRUw<qLW;*t4)5~DG<yTl665FX zMJwJWH9t`Tl;dQ3UfvsCaC9^?o8I*NSk&-?Gf>j)>d?!wu<y}mbhPHL^AXWoS<$l^ zj!+Qfhg|%F@kq55!~A%dJFdwFzx^JTMJpf3jgFe7qYbs9c36ki5=j_YMr9ewPRS;2 zoz@5w3YHbrKSKTy06vUmR<bUhetQkLjj+h(e(ra^#Epo8qYzKmBCl7XXGYiz>p}lZ z&W3Uy6AIs5CSI)E&gf|EKGDm~9IKXR|8AmwiQl{Wci2DmcQ!rq*=3u<K=t1@Dw(1r z(Gx#iKGb8;h??&LIhS7R59YSxs@PZJ#`Z693P1HfhaJ0+K%xnT$HzX}<v9IBUrN>B z+EOGH9@2TZi4HtFv0ptLtIO0Hx#f+2%yYLjvDxaJM^Y5QzKqx7S`$EzD**(GVSlJ~ zd{0g2d{e>Vi9_J8Nrb)7e#O4zU$~}{pqF1B%#P#8>3f4JESFvumSHDF*k2FEEo1TC zj=r`k{t=^zah4LogY9}()*fl`4#bk0nws)*>jY5w+nEwjQR1-7L_FtF4+}krG;`be zS^{0Fr(Fgshu=r5ueVV+C_ds}YbU2#thV0z3|FMobQN`b_vSmNH$KrYFj%a&xf?2- zUkj>Td#mhEW_P+>Y|-IF3oVj14<gCYo=$K4s~e7DeYQ7CP-<S<uG?j+@xn#Q)i#or zO{)D=N)xfWhmo+U83~sMCA9i7I}{fe4xPSz`_}s*rLNPU`pCa^NIF3Zi5#NiGSi}0 zxrVLM>_|4k(jP^MP>@BU558K0Gh#m#r6hY<mS9UGrPV(ptkh`tamrWgC@RZ8);*l} zcozKjoa-t{M&^vQZB^1oylXf}Qa<ln*=*psIU^rOW&7&)K`C3_x=*N0j1=-bpn>C) z3ra~7?hc=)Ws;LU|0h>g_rv%15%D}UkV_2jAWZh>|Cc*(L*EO;HSNq8`mmPUK7#EW z-bWp7M2dJsXE2h;EdKv0vvn31EC_x_#Q_<rL~eJ`6&jZtNS7*j&y7WBDYc8J6&|&E zB{3EdfpElT)K)xiQLTOg)|BCS^l!~x9sQ4c-Jukm@P3Q><&}DJr3@LG@JN#W$l0MK zBovg{L#O|^*FJV?sj8~_Sb|NMxU{66DzgXHj47Tp`#jL!KkepjASx=_*WVwRVx`Op zoPxpSrKOf;W(48CVmCwoGox6B-=Mmwv2k&wJ|#h$C3)PWb_MK|-FwmX-`(M>JWg+L zw6(UbnAAEuJ5NWh4hBkoP%Kuqs81Uhx(m_o=t)n-%oaN{WN#LekkD)QQ2Bt(O^n{G zy}r1JPfTpZkq+|QD9^&w6F;sP+QGxdo&kZ(8&}-38dt!?gK+ThilQ{}`2U26tyi1u z4@zVU6r0_zOjs8}Vh}`U@r@j|-F3@VU7elXUzF)^!C>%bR(TrM_w@0@vJ8n8y6wxg zf25i4eD`@aazkZkz95en&1f~+WSU~98iU^Gv;RLL$5YpEsmPR+l<$d%BE=024f7k1 zg@<5rW7$)Yb-iy?f`u=RN-I5DDT8?F-dZm1vZz0DYH4WTC|5lOw_sih{yV>fkX8+O zaVM|<#k$xEDr#O%L}6j!^0Mag=ugbbe6(i4S$tVp*)LzdtXIHM>=ZUNH60Qs7u_78 zB#)|SX&r+#F@<fuzZD#s+x|@f0h!cJm%&kV6u!-YNv(S;p6;jez?wCiJC}Ee6s{jL zA$xV`BXtV_m>Zw}Ex)Zb%l~1mv5+<wy<=2$YenV<Ki_S1cES|vhq!K_N&l0c(VM!b zwtyO_pMt=h@%~nqv`T*%SMYzeU@r+A7ZTZI#J_OfyUN1%;?8R`OBvh5Ybxj{#r3ku zY#yeoC1m}VFnKBc@DD}lrq*}gl@gpS<wSl_1<5#Ei);{VKah4!U_7j#U!Pm4{?9;| zy71bJPJY)|v?$P^4J~N)xVy&=@U{7o@|5wpvgQ{N5@F~rH(>?<Kuie3;By-y1x}#& z$hN^xufM(y#mGPo2SFsfh;k6t)I=Ej`!_7hg7ZsoI$5`Mfd)+ZYiw~-ul(q@HYM{v z)@1v`Y04$`eM0SJyU+-BCI(KY7RoZp-BIH`YK|1d#pQHWd3)L1#X`_eh71UP4%7W{ zjp|Ls5hg{AOc)dqI~^(WpSE-)jQM&FD{zm20(eF9@fjMHFg#Du+J5+bB;d1Oehv0a zcxb*Opty^FEFQCc-XP)3gW@6v4-^nN8ch78F+Jl0-9HD<%&bm5XnY;?WPH=Z5C-<q z()Fmsm*z67jS9o$G|o))FXd7o-Sy?ivwrFQ&ZpA4ZEpMRS?Hh`h^UaewozNc^B3Rq zn7pgmnVDbuip$TGGmGVDti08HH<tgzPaUI4#>m8=u%=_Ls5YUsYd=2{I+KxzN?qLH z6HyE96+VL;eRW66{ob{ftZQcF>{z@2vsq}xv1PS9a7%SI&AQQVtOBuhik^SK79K!` z+r5a1t6};Q1qUGdANJDJ-*(*_71D!W+dXqy#_C=&&;9hG5_mQYISWk6bln|ew64B= z68-|?+wAuT;PwCn4}8Z?2s5t)r4(MtN&K!EvlbJ2G??Th6Fj~ggVQkCamVSSQGCvm z`TfcC_0(xGZjaMHjV_ddnq5{L3XIs=k@pG*@T8@c{s~8h2e&@|q39u-6g*^oU8V$j z^n70}jeKU^&g8XTvL^3?T3PkDy^o^i8UIOEk9(u<nxAF5)wm{XsPRhMeQcv@UGrA5 z#>zr(_MRI@&#BEk!TGq1<wr_@T$iHjkyAU`!D=tt3S-fk%ZU-Mf&!_bVli3H`^Qk^ z^LX@uA+4;$NB5UR-qdyyN%u{p?eS}$PyU&OQ#n)TQ|IUBQ~2np1IavtjU!F<gDw=@ z$|GzuJL9>u8$&UG55RLTJG8^8p4u%_$ImU=2DyB9xH>a1u(vQ!!%Xudyt%pgLebF0 ziku55=;T`?Qwc7_X*LEbu6R$|y~McUY?lc&S5N*z+#tj4h0fA7r4o1Pv_u#3Dto>L z{+1!}@#vp{lUH`N_ItK+ZwkY(w+(D)sC#nk$*NzKU4wLXbF<rb^I9I!x(jN`*f!;) z2H1Yv93S&!l^}eu&$_l}#k)as<~3y-Z9;pNKx5``p7o5nUD(@GmDu13zrhwvb!@N| z3BdMaw-MO@5gYp&M+p?3OzzzOTw(wJD*4K&xSFoZ+qgz>4FrN~aEEkAAh-l~3lQ90 z65QQAKyY`55Zr^)G>yBvG%)0S=6%=9H?wAabAOy4y>6YZb*gUF-skM9lh-cHdInD{ z>YYtE`^QdWEMB7Un2*9?uPAAtwH`JBTew4>d5(B)f~k?z8>F;|Oy9(t_)OJKE(s9; z+lv7H%oSJo4B|Axa;ac^OiWx-+#tJ*r>bHc&~@AJ0|sCF^-r0~@KeMAMTgs#&1||8 z>S^-z)YMdXcxh>Pco{cY=QO_Y6YJ{E+axdA0;b3pQe$ek{R}XQ&5dOP-@kL))t`2M z|0vKhP)_AyqS~>)Sgn=W?8H+D8k#+jWb-&P!t_mizc*OF59LtW9HgtaDpgPFXD@OS z#CT58ak*HZE=e%`D9n3x>jJsE-QTJ|&^|-tM9pkSOgM=1@sRg$$S-o#nfMf@Z?n`; z^Lo8#cBmn_@<iV~z13jrkZs}m*v4Aus-+<ewv^4x!s7Q<-JAj53#YcTS#S$-6aKUL zkgW^Yl(9e>IO|#&1yzNAT7CFQf-(4;(4;lKjYm*W(8KDoINTui0s!S;&XjQ<2~WJ? zB8P??z%+=wJeJ$v4-fK^mnF$1-a7=Zot@K!Fw(BoVxw1ZWoZ{MurVy?e8woee%QaV zbfcBCa%Hwj+Ieo;emqR}iDq|Pzl^~+l0=M*60wB0_qX^AML$zduS3VkghLXHlUihX zfdY1)-HV|QD;Poex2E3UtEXk`$@_OH*V*y$CPiUSAEz>5fk*-e{4{}doY~#nV_vJn z<7&1;Za(fMKD*M=-vVK)CXf9IX9@3pV_`*wo8VxiZlgP&C>Q^Mej2m!d=0Tnpc6w0 zxj>4~R#g5pjfV6oH6sKTZg7V?cyAp2j-cAi$L@r=$hPmL{~|d{2z7ffMPvJy^$jMW z3QtRwFFkN>e7-p%Iy+AUU(`IrA+=heF~6kIaiVi6K))_E6dH09rupcQuS=!AJB%+D z__+`gaX^R3%dE;v&ekaX+C|9laEAo$GZ{l3GN0cP$=y4|pC+roXzGt}Iv-Ub!#!6V z#<k<a-`62&@oRv7LUNwamZrh#ZM>#qozI2J))L+qc_-76%=mcW>80{5aUz48&@7CF zH2E8I5Sp0Gr8BNfOC~$mmm)h>%I;``DR7oF%H&WN|8M$>giDmgajdEiz2;Bo>?WNB zQ6M^7=@^&FoBSll)y?9Bv@n%bIDoYMBRWf4!RibVKJJi5Nzu7Uvtr*QmFreM`11c? zCm~0yF0b1IQ%0|R`Toh*A6@K+;xo|}+co}hRe{zQM_?i%fq>@&GP$9gdfyo2&N$<$ z3-Y{nicIzPZ+tc(ggUO+-R#fp1?eeVXVL2uRZ;V#B6Rs*FlF#yqN-M?f^F@;iw``z z68^K}C^fgDY)ddYz-Y56O7i?qrShlGpQl>g&YnQcQFctV{5FW{6%V%MTRdFz&mO9J zP=4HjY4RPtuL;VZefE0a7nmcCRfzrIA;JpOqM|LC*_)OEqAR;4jDh{*m&8EH)Lw_C zI4u%a!9F#-)^+c36Be9^Xj^Nv(>qI1$9n_`$%0?cIMyrgoJ}TDwEU(kS@IAzJ9KAx z9Bn}1eHB^|Mue6Er>DOQ4K`bYOXg$Rj(qnH{C#%!XhnErhn=NLxe;(F5AzyxV1Z<% zhIVi_o<tKe%Ayfl;Kh4^qwtG}VOqp4XD!xSbuIaE2nsg?#vDmZ#uHtu0VEOYGZ8^g z$e(P4;SPiDi0Ey^6C*TBZd)#TBB_BIL)EZ0T8a}QaDN_{+9Yckb<spRolkU3+4nPs z*?s3n5j_o0DuyY}k{IvO6Agk^3b6|6*UBhPU-Ytv`91D%-aa-iif{s@oL3+*aDZtE zyib>THR~>9B;;F!@4R3XjlYa8UOXK|h*DXB-UV994>P%cvy^^W!xU{+T#pulNs_-J zWdNwomWU%ns-m*hs~<4^KcuPB;^IM{JZz?B^$y5Cd@;aG6vkn;xQFL<a63kNj1&+& zfky+PUWiI^YN3HwwDrULccqfHOJ>%Gs{SqMI%*aC#tw*hzK~J=>@Y9-1+>FI0k<W& z{WANgK||A%6K_w!^7f<{gID5rzru09s>7%=$vfC-KZC@m>>xTE*ozr6JVAU4xj^Uq z{f|uc5d>f({koFLR4d;O{ii|%nV7tXk%>tL;%YxKEEvnNLBrj#hHF;zepr5G)~Mr; z*<MCQCErKC%gfA8$_R0!!&6g=S#8h@oHH}+MmZ$H*$IXdwWf;M^kQP!#;n`FSoinW zHaS{^Wr9VjM@qj~6R7no#ta1C$!8I|^N+QF;xg{gD@FGQwitRx2Z{(QK}AT(zmGaC zZmaY#YQ#7O5;4A4@Rj+lkEs1!iwdVG;;q3}#qvtL1QrHh&08{41vGDOm4T*k{9nw@ zM~$Ha=_5V@?wWbVpoY4UpMQ}4>J#hoFNe7afr1-(iL`s_1fcO}2Ylx`h0PLJe~`LH zMF|DaShCb!3A(rH<P(5KBUOX6OJad-4vM?w?W|!|z*>UvmmwC0SmS4nhm{~Y2olgZ z@%3V1{*H)*0T;YXQJMghKeJ%_{0#$mmR81V8;RlDR)=Wc_7Me*oV@LHo1F%rj~Foj zI96r-Ey?0LjW5KxsyPQ+tn%XuzrxRmM9}wxAvW?Cs6{3K+2vBUKEaEH(b>8Sgw|!0 zWy_l8R)@{@peJ^VL}Eqh`^*CMGYT>=8xg5{^Zpx$SY^G-AVRK*Z%JES&Ertr9(aQ; zNvBkoqB9hl*vs_oc^}YdPz3<%nktXq7Wx4oJ)hiH3Ky-@t@|fhC<(I~0S-AcB9ez- z&pPNY5x38vd(afPzaT+h-ZgXRul>f+2{qX9=MeyWnQc{*JMgK1>gZ!M(U?)9Jisy$ zbEFv{u!StN0NIaX<!#CYIP|EC_0EDogz8_-;9y9%zJjFVS<+AODhy&Er(0@o6Gc&> z{Xg94oi!?VJ$H&>Yo)*kvO;5!i-ZRq*V_j`fcapx6ODLzQ=l2JD?n5b`}ypdqaGg& z=REoyvE^q3Yty{~ipLQYr&2O8wX`xqUuK071nbeSsH8N6p!AU$%>HQ%^yO+zvW&<Y zy<u%GiN0#@7XbSB^n}Pn!G8DlE3bCjsP+NVQ&{ep#Ld3&?Qf10qBd4FHXwL|5g)UC zXz6B}j@<YiCc)^-lX45EEbbP+*!GiP&DD0>({9X$rCLt`SZ>tELI3C&?B7OX6wg`? zyely5nl`@b059;2*+D?Frbr75mAu2>sNxnK#6>aYQa!CLZ)~sB+e>ZC@^99){^mSn z3jaAUu~Z&#^y9C*?EifnU@~=iJ^lOg6Q{MuMc%mktQ<R<7mvkSSt?p<)VFf`t;4cU z+Sxd-*VNcbSjiNmp!P&0Pf&N$GYc}fKgf8#=1nG4%NSfN&tA9`yKqn~VBeW`(E%9M z<;X?v^_%1yPt2&Yclw_dX<y8dZoVy|er@#q`{}*?f`Jo)t>&&gIjg&yiuZ>#>@iL! z8q!2#y?*#a`*7nz?G2lnVAqu+;wCp6UOfP~wxhmZ^W~_ot*UCOtFjD3pEI<!wr-tD zCNJ?`W?XpM5cW6LX+9CAKTp3GC&{9%)+_A1jC(@N;h)ISiE;7bDL$WtAd13G+>o<( zeZ(4%XO$*1Wtj*?>+L5a@#1xwShIsTtrj!Qms;ENHm%(%QF14;c|Q_S>vIH>V)aVq zR-gRm!0gC0#XyKDQJ}$U&9l#1{{Yv20br~+71u%ZMVYwf3&)}8;Gt<k_djn>w4Q&) z1Vd*`LV#FZ)|F);o%k(cr!V+qD}X6<1l-rQ(w^GF74ynTi?h0Z(}evbpdKPu^Qe6b zp%*DB{6w^XS42~w^@N@|9GIfBS}$yc{S#xqbz??N<#Yrz{By+jy;e>2>8TQN;HMEm z$8|A+Zg#B6$DZI4roY9DzdtICLEV?%-0^@tk&rh~@tf=+Cs*ED>E8n%P~Yf{Et-sS zZyeC~53#ExJ9|OtR}5x*OBqWSloPE448Li+R}5QQgF62f2cT9(8#bn6teI)f!efiG zvo6+<aPE+f6<|qItN!S^0ru6ZoRM`z_Tl#`{C>!M_N@5!_Bw9pF}n7~uS?_4?qg<} z$9>Czm)E5O#JT+D+kEe0TKK=U058vOAttoT%N+#{67_G&EClB*U20WF=hEvR#Guh& zqrQrsSHt5vIL#JAQZDuRs`q0qB-Vnmqm#aO>#(KCC2ey?9W}>4N>_JD4s7k?OS7|{ z_HNF9qEffY=jwX=BT)5x23^-L#`g~iNyt4NRCKKu;D{G;f4gpD`qfRv*)qGXy7t`i zH{@Uhb<L$h>h31hp)6^zVXNTrh@>F@nDi{_cboYZ`R0XW+MS;}=>0R#<0AoMfgeMq zeMm4wGt`Ic)k%5Z^Wc7d+q~TUxKx7IFDjR|Rp~70wbIfqPrvnM6<TAnx=jB|8D&os z4YiCEacIvggj?Q-<n1GOUd_q2gl2_IAdVfpyG&u@Q^pyrt~t{@Qx!UCnNi!6LfMqS z(KQz_7f{w!`s`Crrf}jbz$FtF&%bgS^<d@Ve$v!~LUTO#^FGDFC4-*6O=CMxUqWgF zLW-ll965Q;wVY_W8ZWlQs{G_!my;&PP{(;GB1u(fV(vCn@yGtNnbdTLsbBf`EsCCp z9v@Y|`w#o2%nLp!XkoROt4#-PpaZi;J>`G)CF`mMX^}}E$4-faH^1a%|A08(2y?E^ z>fNoiP&D%WEzdm6r)PE{qBl`+DZR1)L**TYE|Yy)3@!$-MIR1yl{HT2Wc}nt&h%7v zOf0G&4`lPMBLuo{;R?zNT@KaJ6{2_WkLn)Q9!q0Lk~Gq1kTI1u;z<V{Hgj(U)t%0Y z;L+}zoB?xGi1F*8%#lW?H=_>N$we{Gg*v<P5{z(G-wvC*OKPxCax;(Ew0}+@_VaW5 zwDNAxB7OPJVcQyo%YF>_U~9)tjBrT*0qDHy#KS=Y5T2huH(&SW=%>WgsO@NUO^he7 zk4F9Q&aL!QtzG9yXon!EOI6G2<0A9Oh-m`44FyD3@lwV6(503fbr3dJf145jPpAQ` z_pXG2@kF*+vYuJ{`nH>ist9@ua>=#-$y`dkX_IZp?W3nR;zip<OkjuA1fUnG-W?^( zoc9*1cl-u=wiT26Tj=e5XB#O#orCGJN1l#Y>@H`B@=Uq9>;^9LddOmCH|tv_l3HeD zDpj$>Ili8kYfNh8+Bjcy;=GtCgUQoysV0e@7W1BBYdLkmo3V3@Q>D~rzO~~uuf{05 zlC0rI@9KWvl#=MQzLeMQ9xCq=wS0%xslRE9hROM6og}6q(MMf%3KNd4FND*QpaZe1 zD@1i9pR47>Me!$gkxJ9jKqhl0V>*~E$E2H<Rcb&)+gd3y`8g(D_m{y(yx{?97{=og zq$Qe^(S*qS`pe`Juj<Do<|$L?jJPjeCseKm*LT2%JtDE^*U_umO<E4DZnMN>9_ug_ z!))RNn4os~pll*XIG5w6?WMu#j|1c^%pmr2CnU;*->r%o=^g8j4K9z0vQW3BPFT{L zB+?U<*l0W)ZiOj~M0Ui3DdG|qgI)>e$k>ckf!ykX;Zf?TQAK|FtY-JfbFxKcwM!v< zK8I$*F^#!r%~l(hbQx|R7j*O;>(BJH%&?i1m@t$Lm|r6u1>nR=dp@9LC+ZRStb;`= zd^CrzmOPK>)-%{(9%pS|0u3CA10;4I4H9q~Y$W^1#xo>d4QEKasTiV0Nbj=s`DrT5 zO2}81s3xVWsC2scc?;pq#5B{bO6x=??4)vb4X2B^R*04-u~qlefSr`sX`_2k4i9lV zRD*$d$;~*lJ6#6H!~H{{>*V_Ehtp>FyPO%~S{R3N+Yjk;`_^wA6dKGB!6+eP$E}z4 z9Sr<_E8dYwwJ6=!kvBU}pSBIV?8VDY^S&02Dm(UQkXR9SGC{8jf->2rvD6ZmNd!R8 z?>E3zK(~p#=*iD1wFydXr}Te8obwkc{QuhujWK_NWCa%kgCD<eILI}l%@&YD&|c@@ zjAG(U{MD%UqXlsM$4aUP*^7&-Q;58-#veTzITp4DNFu&me)NwHVoiso8ITWZmZ|0> zH}GQ87=Kfu$J0Fc)kra_?e9ci!<-fyi?gfy0(=-98~dx;ve4rt$I8RT%dj4V?6NW| zM@OzHuh(xD8Sp~m<KvfGI4lD$k|b!aJw>AWsL{ZbFVLLy`pNEh)jDoB*nEh0b))%1 zgM-J-xHQXn();?~;SCyp(<iz$p~ZZsKK6vfgBKP4=GVjj|7{#HReI6~I!yHYS8>SS z3F4?Q5&xG6{KslgZi`cB5N%&2Grw^B5yg!4)B|s=CVQH(vx$ib#so$I0nOZrQtc*= zsSGOPL;CmuH~|HPC?6j^+IQFyA<{d%p5)}Gz=uXR<tosgL6+J)+%mJSW43O*?k)o= zMg144n`hsoz~J5;swLzkBp44w`1sQ>n7oTP?HOUjle?cvMfFyYp9=H<Jzi+m{k~~@ zzp?ILU%Cbculwu(zDSRi075X79DIl_s}~(W`Sxv9P0gfXm7}wBU^AQlA}$F-T)=V< zp<Z}ssQXC|;d|&y@FD(_Yw)sji;<Dh-R<qUyI0{IxL^36*`gTOj99&Vb|dsEiW2Qy zMF$L~qM`!lPMFncHoFFbiFI^ztgNhZb8~YJ(O$2}<tXIk<*73#D!9Cf|K)Z2qokyy zu8w=EjnaQHHZ_%Gw25p+P4C*Xy}rI);p+Pf`=I)Io~$v#U0r_}^`Pp$wHysh@cI`= z$Emdw|7OD1xHCD-XyC#YF|-sgUk7iuK>Uv8(+dp$Og#S=RJ?e9E%iV4d?Uu(YN3{4 zN+-Gw_HfzY0ExyBtl=k2{5w*d=w~)F!@dT|9`orKf*Bsn%o>2%U(t`TKRD&TZ8h4( zlZG}<UEf0^0)Fk;HEF+kI|ZdQCcqnnr8cys8tk*+9$DLzWEe1x-B0v-Dw_`_O}k-r zuV=K>={J=88hrwQC4^<u&b_L_PLjc8sFvZqW?UprUuny#b1Y_b>*6_NK23gxS`8T6 zYoZj`P)d{6eZVrZu#LFC?o%S|Cfknwy|c%l3L*B`m~y$qfmm3DYnnbeAfYYGesUn+ z++`ogV4iB2anUtn-|KE^dyDckt1vcfMvf>zNRL*cn4Q62-DFKoU2F=BI`HR?_74wR z7gMru5MPRoXl?Z>sq6~W^nsR{Iw*)<F273ICC+O$+%s@cxvG5psmJFrteKnu<IoC! z^DwpJ`EX+X3eU5KD>@@@5pg3WLPLw`LnlPD$y?7T!NDgMhjYq7ymR{H<7dG=M!6-! z@~=2>-h1%Y+r5+0kw*cDZ_1s8>DgJ-nX>4d0EaXp_z)CuO2(oT^_ZoZ4+2JwtCF!2 z#iglA6OrNuVvXP05`j1zN=^Dm{1}!?OPa#Gp1W{Uo|%_~D1!$onD~@bs<}x3A*cMO zc(aF>{U}HDZD^aZW@D`8tp7uBC2`b#H{m2^SxAo-IeCC8*PN#%h|Oars_{E--jb(9 zm~G5jbes{v(8FAp<Zt`J^7lyNx*nanww<Hw`7E!=&`F#-bZo4cJ&R7)y)6vBxJ{lF z39(BHUnn1bYFz?^taJvLa@JXpD>7Ja1j8_rwZ|aO7BA*I?r(GVmYxO;nRM}kaL)pw z8!~DrQLl64rd+>HwnAoFuW*x_(diZ6q3iZ2WKO%M_t$&=$XtqM;CzZe;F1C&3K_jI zeRgQYJgiToHIWl~MN%&6Qb`%o-SlUxdM~q;`T8!S<<gi<bbjUH4*E<&BkSOt^o9ZK za=@f^`#aHRbm7d&q2uwV?c0;%OUA7-A~YV!>xrc>n-0Z`UhDRsy&!3_qheQqCf19> zWeVp*@a1Wez?*BRRpeTbRYBUBa_v=trh`#sMmZb5ZetsB4g3HqA924%wmVQjwrIgJ zJpQ)6^*#PHwjQfsqX%_R(XQKG=S@k_Q2F^yK}7+cr7iD;Bkwes=%P#WO#~N-dR&WG zj*!G<*4EMR*w`2>rK3mWea%bC1-ST%+*TE*j?ye@km6|#otigiBb`QqhhyW_sYV~$ zY7pDHv56Vc9e>pIe8@)&&Gmw#4&}!t2u6C-F+Mv&d`G*7QS3Lw^CJD5bHa4f20eZq zWldEL6x&9tLM$&v^-+C|I>w?kV@2#sUC)wEO79y|x<5ZEIvz)lG@(GG&jQPh1kzs2 zCX{p%d(n%2s_n(?9jMc5ZSQ0)Sy}jIF?Y*@jaSh7<oq~*w}u0U$Ri(9dFm_@Q65=+ z`19Cj-yWSQ2Rv&sb8jyrccsFiPn)P18%+bli~3roHS?BxmOcq7oJNzR<7M}5A!`>6 zFt@XjVf(w#?fJ_CE0${fI=fGMJT@LT(}QWeEz--^%YT6L@jWVyN-=mBu8OF)IJetm z`BiAy-jxpC{E^LEEn%`fRdc=oTM1uP(c?DbGX;>ZKrAvQkM}`HadAGQd2s}|6XMO` zCavLiKaYb)hc_$hxCfb0wiGoVE7^?w!)wFjT35aD<;a|Gshw&yXW9V?SXWY^@~O={ zCA+y7vt6gYF(tmby@xAnIdjq4*!(UU{k(CU1zdyhxon-CE*sCiJ1D%TsWJvS5rKkP z?nD#3ii7(CZ9gUZIzduUOXy_UN^kW@6HhRUTWWe~Sa-6c<`8trw@u<X!Cb43Qw0zb zcl$@hg`T=&Q_{;cRrB2SO{ShP%G8i|v<0V|CZiMoqtnDAeD*3w=t|}*EUT5A=Ay-F zuhk}tUT#zb-hJrD^D4PM6CkP3+q3Jrx%Y*2NSg8-O98DSql$4V>2W9WO+``zkfnX3 z?C)Rgg_C4<d+rOj9G+Lp7=wPqoHqPa`hbuMdHjGVUfe6XM8(~ZWfMAu811WaL!+u5 zC1mu?Us5br2^13r1eJZs<?l+5j@DmtxA*+@=e$F3&n4dKZ81@xB5_>J$i{`>Bz98; zcNxQ|AyMFjaj?X$OX(`mDX_#OMiKIRpDNzuEmp1^U%g?FlYH--3F*GNU&WqO7-)Y; zN}+^X0O~N(U7~AI%s9%yiIqD<FQoMNyd+)$bgS@~ucOvlhjk}-1p#bJ>bWT(is^L~ zAJU65W4<igM$Vhbv8UnU;MA+~_g$J)I?W^{Sylb*RZ(1xeEe)j%&0@$p4;6w34|C0 z+43z@YO!5$znGRH(>I+H+d+%mwCP8nlbfw9pC>dhDco#6`7@e8;`IFd+54V7EO=S) zd9(J9*vYRqJFs=#C~5zhY!xuhhmfr!D!qTO?TYmR%3CZ4wgx|pcLp?71#13MWt?U< zf>KRW4!_#)oL_5H8TJ@O2@^c+@hD!tLy1Zj9J0-~-#CZeFC!)+c@<ANjYr7Sj!~?0 z^oX<9Wq%IzNnX6E{+Gn(-y8dX-=0T^fPZ0H$Uu*UF$2pz$LBWFR!@55EnRscF9h}; zIV;_}T~_CoFnzAKcbB328aW|)Rd3vn*>q-wx}`I(oguO4Pf+2BB2DS2Cl^+}<`!5# zySWuors2ru*;W_BxedaP(x2NRB_<+jAG#+|`^g$*bcfPX?uHiwv)AJI26oe|WgSKY z2ej$og{^hmL8t8|$PhpkuJ<(#EM6jo5ubKu^2}6;(Ima}YCZo<6hzAFq}>J|CrF#H zm!Lk}#p;!yVYQTBAng9Gt;PS6frEA(_~4n-bin3Usyl@6$Jaw4M?NH!>}9#rGve6? zTW&!Cfy+F`fRc;uGsL$$@}-esUtXeNg?SN*+{9}&(f0HsjF(A;x#&!wHe7@w?B<=H z<J)PPAW>rL+^%;^dx^{>S6>R!etLnhn`~C6gwrDWFzGgCCRxHj`uD;Zp?sY)%a7Vz z9=R1`VQ8!i=gDqW?w{Oeu43!bm0}U3Xh@UN+x-KU$i}MD?RJz{2;VY?|L(EiXB|8u zm(@&bayMrtJ;uI6kc#x-a>^$e-^&FB-```%S~8EFgb`MeSaJ0gQB&vs`1|lBo<)5W zjY}{H-OS5*jXqeo-6*kUSSwIwFC=HpN&->+*0UozEX8A<aD4MdjJVEzM7ausC@J}e zI;+4;f5{UprJCE$k7Cgx8`K;`f5Pl%vF@K@34k{%CFPexI>LIoJ0!u9a&JM$+7EdT z&$r4wK$UiOZ*PeRIu(xcYc4J*TqJtwLFS#p!-kc5cz3MnciUmFUW(~oTTtLF5OKda zKF3r<u|irV-zML;xVt~{zRF_pdA20{0e)ID*0vwL)f=%iMB(F^V?z&6vJ2R^-`E4b zt{YbrTwY}q5Y?Du!&Sg$jKce|3)P9@@)Iy6G_;$`4|pYsl-^~pG$R4AS-k^BPna~? z&D^!~-~$<|^UiK~zVDdB#t*fuUkJX=H&@k3s9MNEh`T0aK-ERAG(u-@U$vNip`L;` z%WPrpb>XzdENa>_?DHsaNkk|RD3qyqe@YL4j{x5u=|3|*rshcFE=@&6ZBg9iPYhsn ze!H%Bc~TM)(Jz(H3yGa;8I(PX+<>1*qX|0KZRT^XX5&Nq{QI{!sL}2w7Ij=4@U57Y ztr;_-k4H}ldYa%)xoT`W7}q(xDQmTgrOxx!H22JuZmf@L(rV)IOE22cKvhB0A}IQW zqOOzGL`Gb`uU&jRH3Jz^&n|VXZ%gUS4{q}nU)-T&e_Eynj<Bx^7m5u)15nXI+1sTB zV`@sToB5Qp*+7u&EiTgyTG1AGvgg~jlSSOT?;M+077lXuBU_uVrf9vDSQ-vUvTM?7 zm*!K|OW%g0W~n4BeaRO<;YC-Bhp5RZABG8d^YsUr;^{#IFE!TlM)6!8Mk@Ak2wkb` zaN!iIRsvBDmoUjtm&<*PcLEQ>@e-P@MNV*pA|latT;fj$AP`kTCd$Xf91pO`f;7yL z<sx~4v!bT^D#gU`2=dEhc@O}Eu%tZE8t~5Uc1Q?-(Cz0faxh>_fQIEJq@Mnw8v!&D z>)(S&-DP!$8l>_bB+isnq0!JNsvkay(CEDrD|O|{e&gk<Sz!wL<T&Z%fJ+}=Q1`SN zE4aLyIoxnoxN4xo9M-{~aXARvV^Zz=&?C|;Kp>{SeM+|_{7U;!t6C^A%;Q`USiJ~) zJc_fhlg|T)ww(}xsW2*Ppu$5R@FC`4jrjA_%6&VK7hpBy0)B6E11RNJHE%=6Uc7+r z$6lWl7}r%X0~4ZLl&@6>D&V7E82}(yfqwbUoevP9S}<19BSo`j(*JyYG`2QND^WG^ zlMiUm45Mg(yV=g;hJmoon?S#b+<^Coq*rzRXQa?v`TDEvxVjscsCKtiEqv9S88g0~ z?LxH|*7ql4#e@#U!M<ul>9+#wT2o=%jMo8O{4!*fB-n(<&#-R~`2_o_nLkQ+<-+(( z3TApg8cgx7zx=W{!QvPt0`7-%3NX9U(iGdcj=Q+L6%?W$cA}c4lfkbPfyEaI?(gKH zui1QK4Wa&D(jVUat#!y)Mob69cU1*bz3d;3u<O-0!`nWjn@xW;O&*--_`Hy8Q<+d- zt4hd@rEE9KSvBd9?;L1QtkpB~RnVw}pyMVFGj2}POTr|y*yBwnBz9oER>q07BR}3i zD)q~pfpkxveSwPWmpK5)yA<Vd4E`fbhWgqi`{}P67kw-TEf*Vde;|ulY?OON2&U7d z+1yG1U7{0%M})pFMXz|dGcur1xjyes_rdvj=)6W;*7ukSVBc+TQ5{E8=dpSWr2m`E zK-|~k4PGBU4niet9twd%p*uoWtV1xBNfP)KLje!2&#t4$<26=`^gzkZ{{Eq(KxwD9 zgoBNX>DG?W0kc=Mh<cXODP}4s%@uaQ>H7k&-GkfMUSZ;dx`Y|9gq!jQM$yZDstZ@r zIw@RBi)wY5pZFmJVvpMxVFQM5iTQy3;dk@s1%!aODfA@B3#;0ou9TfvhBl7I$!Owb zN$%C2*7Fc@eB`ben}u(2EI7*#+fWzL=4Uo{pq92m`X~dUKnAnzQ#s2nb6JA0foiI^ zFN3AE4cIl9X{QoUv=$h51t^`!D)Y&U_YlUPz~RW{;7P~h&Z}&`wS3j1_0g9cgpL;< z(1<MW_M}N_>hU(?c@Mn58q&`+((#T75sX8hb@Dv;`mBnQ0oM}#WOur0v;~*)75NY> ziig{a^jQ0LSNZGkXdAow$D(|0l!qJ(ibwk_hrS|p{r%9{L?7Mut&-x0H0;4kP1#fF zrZkdqox8#QK*LI_XMNN6k-97Wk5H4*;zjo}6-b8Q1ttK;@pZ@_Gnt)d!&GS=g>9D( z8L>SlY=y7+@59$?TArB@+K58#&ibHw<aXy-j8dDgE_31nLc#oR0`KmifIsQ0o%-3% zhcoyQYaj(9mba$zBwzO>*w|2RWz;{ElcxUY6;t;Zn}}2Kb=Q@qGu5~1(AW|H@V*QA zNoXIpY=th8_yE4r*rO{bVG<n8LgDfjLw=$=00Ku8p$(6%Yx%aMcQ0M0n$_g6VsKW5 zS?xl-_w9qAuNm^F_VriDX6$V&w#HtAaUyS9>nLFRk%@|edy+I{k?p(%58;4LZ$asc zFx1f&mb$-6C#VNC0|CZgtSj?TP_>n0M#DX9qdHIbpIKZ!x@{SUbvN(LqE9@Yc6zY9 zr>=bEvB!(7jIGpmgrDXx=p3IlSG_`axmwNJapQVC_6$EL1%~Lp?G(eTPO<ttv(S)2 zQC8?j(=<BZ_#<kO7jN)+T0<_6+ucd*?O}v`3Xa^q%;*xld+AIb0f4XJ;zh%MF?*c& z-95CEM;<`P1?8Pir3_lzP)5r~UnMOaU3FJz=5Z2V6T5JY@k<2uSvVK0Tyn&xm(8`w zL8cW1X4^{6zcf}^I+SA@N@vp2?XB9%>)5GQ^(yKm#MS*=-FblSNCDLE@dnr1b8x+$ zYp4yJvdsCocZ?wEPuKehC9VzMW+nI)_d|Mhfou?!3NA&8i%@YW)`QxOr8jzX(*^NR z+2w!b4&U@%nP#}FcY)J%>S|ouA`O!LLnFy&db#<hMZ`o^1xCtVR&d+@nJGS5b@(rQ zLdf1*;&SCa@^qwrE2GbOf8hGg`ICvJ-|B_d7|Ss@E+}ZGJ-d!=(@?k6YvrO*+!lr9 z`2drYRAw3k7NDCX{n=iVMHIRjJBw)@Dfsx)gJB;Gw$OV~4cKXR&kYC0rtv5q8S<iZ z4=1V8^EaP=Ulje<fi@0sj?BL-eB4!-HEHPmmfxXh#kO6OHkDUFBs1F6IQh-#S{O|U z<W3Y$vYWej*13H3lU40q;F>NL@GIL0znEJ-$Nm*waOo>sHbETmF(Q?r5WZGAAaqbI zzQ#m6(ro?m^>e(ZwP;)=y|qVs`pvb?ZNR9}%$i2pqf;V?^J*?+6E})2b8})QMuWsZ zBC|P`C1^wz`fDWZVYH?%lCWHKNkH1%R69}f7~0eP!Cy$>sZ*J*Z6Mc$=2$iGfEI?X zpw_7i0X|UG#>K|cxNWQxgqGcQ>D#vKn_(a&+J1gG;ZosNpsr#Xem>nI0c+dDmnWF} zW66^S%}g?FAk4r6Rkc}LRv;&kQvp?%V!B=5r~<24S}n|>I$H)_?)f38@rh2zrcvY? zO=!Tvp48-b%x5ZG!9eQ)>b@Il=)R0*&-8M&xR?ZVPAbFv^8J=}gA}QDLqEj(Pj7g_ zjCGA#L#XdoWKW2-Pn+f!t>w7rmXA5`<r4<fD@{yJ$8eS-g%3{MNkh*Lm-@R@P8{uZ zb``rs_ZI{9H`tsck6R#Qea|6syjyf7UoF0(<}DWs_&|Kle%hWauN<)P+vM-g)^XP% z;ymUe+O*z(oZfpJQdM3PGsHQ%F_N=)-klio_Bks55HRF2E#=!%w0XVKJ~?8O$I{4l zMI%%*w97p8Rz`OsXZMOE(aOWk2<M1jEj9hVAUETC2doGdSGQ-3dhC>zyJ?$Mb_4SX zJX}yKKkxFecDu~%aJq!GGLU)Zxtb<87-s#GmTL5;X!AZfC{CTNQCu?<KgD^_-f@_B zlGi1G-;e5eR3~NaA1}GLDg7r9FOHie#;$a&o#CrHE0{K4Oy00xN+n%y*rZE_wLd%v zc33ze@;#V0tW&%!l}L55k>Rx+e;@S}g@ESC=>(Gm?k*K4S|m~;=1(QDp9{R3P(9?^ z>h2g$b_*i<!w(<NGhpd<h=n&%tmw9Dl4pCQlW_z~)3xy8#k20V5mR#czULh&aS7Q5 zjH6BEnI(SQC%1h^9m+5Ptf}-SKuqk*Bx%cJ92Od@^l#zm1EG>TxTA@=#)#ea5xs2X zH-qV7mEud~fM@B=@EK*zlN#KJnc9ANm`}ljgNv=*4~QNc_>d4|16A(-nwnWf2Atkt z5?I0)a8*>KRpMslkK5HOb+f#Rqat9(xPt5(?nIeiTbk7MmOQwU5VH}oLyR>Dg7rL# zcYmv9*n=vPsb>f_RO;<^_IRGQrLWy1X%}}@Tf_MHK%4bjOoIIdWjZ-49sU`3{TNqK z$&96r&#s{!{E8PoqT5)o&{B~9CtXE*X)28r&PS@3jutHH`eV0mu=C9&03g2ls|9Fa zkrVIt63>Bmp4{?D$a>pZl^}_o*7FL23gg3ueck7u+dDYaBsMl|%}S8cJ#Xk$o1U-d zAB71{x3@V3WsbNQ&!}yjy^88vQ(*~3E@$;vt$1pX)JT4Fq^%}jHxKUGo8k^2?qOkN z11bvc<7n0Sr+=t(l2B*OrKhE(ajVy~ZH_5h^6$4J?E4%nACADHT4sRzJd~yKW>Zd# zGKt-@NxrBdYAM_>+>OnLD4NKFoBN4oRBz8W%oDSv6heMwHSI#LhctOEYwB7bV3T#9 zD!5gMXOsPgC@G4LwDZN84zC3T8jB;H+RBN<Mn|6}mhgtyuB%^Hy-|-Gpn;yc9C<f( z^tD)^rk9ND_r{dVgTJ3Bj(Q<mZCPUte8?Yb&DGo`UT-NS^LLx7Q?J>VOOV)R%qV#Y zWgX?U8d5gGdCamJ+?I|beAu9~+2JViFCG)+!}KM$8Q<jFM5<(yu6Yhba$R-6H79+D zX4!jyXoo!SKE}gUR9NyA<+?~Va>%Qh@8Yp@lT6_J8RFW8rGDKB8h8GsPWi{B%<f`= z*zFV7F4%+kRaz;*8yp?s9~5783);#ee++G?*rOa^(ebviCiR<Rx8mq3em-@n5$Y`s z!-X<wM$-oqyrFuY8OxhOi;UDq+Ni-AlKkXPO1Gs_P4NL`iM^?M8M|g?c}D6znNQM1 zs*@cR_z-`D;lI||#?wz_hyu}_2nlEznO{%ll(7A5yZ!zGhjGR8;-5QS|F_ruo#-XR z{V!H<skBz$OIR>!)a(f}FccZwkN4}Qjd&)fbowhk#)m?WcW6z1;GFTV-B;}(1M)%U zp`Uqq{>_B`(Smt-d6mqhe~U|vK5oAas{k_#3sQ1&Y$rW5bQg>ODjFJ^>vAkDgZqsb zbdjvFwG*`OFVK*{P(tu8MR9TekGT_H>sEw5KKt=6{(%0k>s(_<G5*?@{|y`S_}u-9 Vl+UcjjstjlK7EiAF9+*?{~!6FmY@It
deleted file mode 100644 --- a/doc/docutils.conf +++ /dev/null @@ -1,9 +0,0 @@ -[general] -input_encoding = utf-8 -strip_comments = yes -toc_backlinks = none - -[html4css1 writer] -embed_stylesheet = no -stylesheet = style/edgewall.css -xml_declaration = no
deleted file mode 100644 --- a/doc/epydoc.conf +++ /dev/null @@ -1,24 +0,0 @@ -[epydoc] - -name: Documentation Index -url: ../index.html -modules: genshi -verbosity: 1 - -# Extraction -docformat: restructuredtext -parse: yes -introspect: yes -exclude: .*\.tests.* -inheritance: listed -private: no -imports: no -include-log: no - -# HTML output -output: html -target: doc/api/ -css: doc/style/epydoc.css -top: genshi -frames: no -sourcecode: no
--- a/doc/filters.txt +++ b/doc/filters.txt @@ -18,17 +18,21 @@ HTML Form Filler ================ -The filter ``genshi.filters.HTMLFormFiller`` can automatically populate an HTML -form from values provided as a simple dictionary. When using thi filter, you can -basically omit any ``value``, ``selected``, or ``checked`` attributes from form -controls in your templates, and let the filter do all that work for you. +The filter ``genshi.filters.html.HTMLFormFiller`` can automatically populate an +HTML form from values provided as a simple dictionary. When using this filter, +you can basically omit any ``value``, ``selected``, or ``checked`` attributes +from form controls in your templates, and let the filter do all that work for +you. ``HTMLFormFiller`` takes a dictionary of data to populate the form with, where the keys should match the names of form elements, and the values determine the -values of those controls. For example:: +values of those controls. For example: + +.. code-block:: pycon >>> from genshi.filters import HTMLFormFiller >>> from genshi.template import MarkupTemplate + >>> template = MarkupTemplate("""<form> ... <p> ... <label>User name: @@ -86,12 +90,15 @@ HTML Sanitizer ============== -The filter ``genshi.filters.HTMLSanitizer`` filter can be used to clean up +The filter ``genshi.filters.html.HTMLSanitizer`` filter can be used to clean up user-submitted HTML markup, removing potentially dangerous constructs that could -be used for various kinds of abuse, such as cross-site scripting (XSS) attacks:: +be used for various kinds of abuse, such as cross-site scripting (XSS) attacks: + +.. code-block:: pycon >>> from genshi.filters import HTMLSanitizer >>> from genshi.input import HTML + >>> html = HTML("""<div> ... <p>Innocent looking text.</p> ... <script>alert("Danger: " + document.cookie)</script> @@ -112,10 +119,13 @@ filter will still perform sanitization on the contents any encountered inline styles: the proprietary ``expression()`` function (supported only by Internet Explorer) is removed, and any property using an ``url()`` which a potentially -dangerous URL scheme (such as ``javascript:``) are also stripped out:: +dangerous URL scheme (such as ``javascript:``) are also stripped out: + +.. code-block:: pycon >>> from genshi.filters import HTMLSanitizer >>> from genshi.input import HTML + >>> html = HTML("""<div> ... <br style="background: url(javascript:alert(document.cookie); color: #000" /> ... </div>""") @@ -130,3 +140,104 @@ suspect to various browser bugs. If you can somehow get away with not allowing inline styles in user-submitted content, that would definitely be the safer route to follow. + + +Transformer +=========== + +The filter ``genshi.filters.transform.Transformer`` provides a convenient way to +transform or otherwise work with markup event streams. It allows you to specify +which parts of the stream you're interested in with XPath expressions, and then +attach a variety of transformations to the parts that match: + +.. code-block:: pycon + + >>> from genshi.builder import tag + >>> from genshi.core import TEXT + >>> from genshi.filters import Transformer + >>> from genshi.input import HTML + + >>> html = HTML('''<html> + ... <head><title>Some Title</title></head> + ... <body> + ... Some <em>body</em> text. + ... </body> + ... </html>''') + + >>> print html | Transformer('body/em').map(unicode.upper, TEXT) \ + ... .unwrap().wrap(tag.u).end() \ + ... .select('body/u') \ + ... .prepend('underlined ') + <html> + <head><title>Some Title</title></head> + <body> + Some <u>underlined BODY</u> text. + </body> + </html> + +This example sets up a transformation that: + + 1. matches any `<em>` element anywhere in the body, + 2. uppercases any text nodes in the element, + 3. strips off the `<em>` start and close tags, + 4. wraps the content in a `<u>` tag, and + 5. inserts the text `underlined` inside the `<u>` tag. + +A number of commonly useful transformations are available for this filter. +Please consult the API documentation a complete list. + +In addition, you can also perform custom transformations. For example, the +following defines a transformation that changes the name of a tag: + +.. code-block:: pycon + + >>> from genshi import QName + >>> from genshi.filters.transform import ENTER, EXIT + + >>> class RenameTransformation(object): + ... def __init__(self, name): + ... self.name = QName(name) + ... def __call__(self, stream): + ... for mark, (kind, data, pos) in stream: + ... if mark is ENTER: + ... data = self.name, data[1] + ... elif mark is EXIT: + ... data = self.name + ... yield mark, (kind, data, pos) + +A transformation can be any callable object that accepts an augmented event +stream. In this case we define a class, so that we can initialize it with the +tag name. + +Custom transformations can be applied using the `apply()` method of a +transformer instance: + +.. code-block:: pycon + + >>> xform = Transformer('body//em').map(unicode.upper, TEXT) \ + >>> xform = xform.apply(RenameTransformation('u')) + >>> print html | xform + <html> + <head><title>Some Title</title></head> + <body> + Some <u>BODY</u> text. + </body> + </html> + +.. note:: The transformation filter was added in Genshi 0.5. + + +Translator +========== + +The ``genshi.filters.i18n.Translator`` filter implements basic support for +internationalizing and localizing templates. When used as a filter, it +translates a configurable set of text nodes and attribute values using a +``gettext``-style translation function. + +The ``Translator`` class also defines the ``extract`` class method, which can +be used to extract localizable messages from a template. + +Please refer to the API documentation for more information on this filter. + +.. note:: The translation filter was added in Genshi 0.4.
new file mode 100644 --- /dev/null +++ b/doc/i18n.txt @@ -0,0 +1,248 @@ +.. -*- mode: rst; encoding: utf-8 -*- + +===================================== +Internationalization and Localization +===================================== + +Genshi provides basic supporting infrastructure for internationalizing +and localizing templates. That includes functionality for extracting localizable +strings from templates, as well as a template filter that can apply translations +to templates as they get rendered. + +This support is based on `gettext`_ message catalogs and the `gettext Python +module`_. The extraction process can be used from the API level, or through the +front-ends implemented by the `Babel`_ project, for which Genshi provides a +plugin. + +.. _`gettext`: http://www.gnu.org/software/gettext/ +.. _`gettext python module`: http://docs.python.org/lib/module-gettext.html +.. _`babel`: http://babel.edgewall.org/ + + +.. contents:: Contents + :depth: 2 +.. sectnum:: + + +Basics +====== + +The simplest way to internationalize and translate templates would be to wrap +all localizable strings in a ``gettext()`` function call (which is often aliased +to ``_()`` for brevity). In that case, no extra template filter is required. + +.. code-block:: genshi + + <p>${_("Hello, world!")}</p> + +However, this approach results in significant “character noise” in templates, +making them harder to read and preview. + +The ``genshi.filters.Translator`` filter allows you to get rid of the +explicit `gettext`_ function calls, so you can continue to just write: + +.. code-block:: genshi + + <p>Hello, world!</p> + +This text will still be extracted and translated as if you had wrapped it in a +``_()`` call. + +.. note:: For parameterized or pluralizable messages, you need to continue using + the appropriate ``gettext`` functions. + +You can control which tags should be ignored by this process; for example, it +doesn't really make sense to translate the content of the HTML +``<script></script>`` element. Both ``<script>`` and ``<style>`` are excluded +by default. + +Attribute values can also be automatically translated. The default is to +consider the attributes ``abbr``, ``alt``, ``label``, ``prompt``, ``standby``, +``summary``, and ``title``, which is a list that makes sense for HTML documents. +Of course, you can tell the translator to use a different set of attribute +names, or none at all. + +In addition, you can control automatic translation in your templates using the +``xml:lang`` attribute. If the value of that attribute is a literal string, the +contents and attributes of the element will be ignored: + +.. code-block:: genshi + + <p xml:lang="en">Hello, world!</p> + +On the other hand, if the value of the ``xml:lang`` attribute contains a Python +expression, the element contents and attributes are still considered for +automatic translation: + +.. code-block:: genshi + + <html xml:lang="$locale"> + ... + </html> + + +Extraction +========== + +The ``Translator`` class provides a class method called ``extract``, which is +a generator yielding all localizable strings found in a template or markup +stream. This includes both literal strings in text nodes and attribute values, +as well as strings in ``gettext()`` calls in embedded Python code. See the API +documentation for details on how to use this method directly. + +This functionality is integrated into the message extraction framework provided +by the `Babel`_ project. Babel provides a command-line interface as well as +commands that can be used from ``setup.py`` scripts using `Setuptools`_ or +`Distutils`_. + +.. _`setuptools`: http://peak.telecommunity.com/DevCenter/setuptools +.. _`distutils`: http://docs.python.org/dist/dist.html + +The first thing you need to do to make Babel extract messages from Genshi +templates is to let Babel know which files are Genshi templates. This is done +using a “mapping configuration”, which can be stored in a configuration file, +or specified directly in your ``setup.py``. + +In a configuration file, the mapping may look like this: + +.. code-block:: ini + + # Python souce + [python:**.py] + + # Genshi templates + [genshi:**/templates/**.html] + include_attrs = title + + [genshi:**/templates/**.txt] + template_class = genshi.template.TextTemplate + encoding = latin-1 + +Please consult the Babel documentation for details on configuration. + +If all goes well, running the extraction with Babel should create a POT file +containing the strings from your Genshi templates and your Python source files. + +.. note:: Genshi currently does not support “translator comments”, i.e. text in + template comments that would get added to the POT file. This support + may or may not be added in future versions. + + +--------------------- +Configuration Options +--------------------- + +The Genshi extraction plugin for Babel supports the following options: + +``template_class`` +------------------ +The concrete ``Template`` class that the file should be loaded with. Specify +the package/module name and the class name, separated by a colon. + +The default is to use ``genshi.template:MarkupTemplate``, and you'll want to +set it to ``genshi.template:TextTemplate`` for `text templates`_. + +.. _`text templates`: text-templates.html + +``encoding`` +------------ +The encoding of the template file. This is only used for text templates. The +default is to assume “utf-8”. + +``include_attrs`` +----------------- +Comma-separated list of attribute names that should be considered to have +localizable values. Only used for markup templates. + +``ignore_tags`` +--------------- +Comma-separated list of tag names that should be ignored. Only used for markup +templates. + +``extract_text`` +---------------- +Whether text outside explicit ``gettext`` function calls should be extracted. +By default, any text nodes not inside ignored tags, and values of attribute in +the ``include_attrs`` list are extracted. If this option is disabled, only +strings in ``gettext`` function calls are extracted. + +.. note:: If you disable this option, it's not necessary to add the translation + filter as described above. You only need to make sure that the + template has access to the ``gettext`` functions it uses. + + +Translation +=========== + +If you have prepared MO files for use with Genshi using the appropriate tools, +you can access the message catalogs with the `gettext Python module`_. You'll +probably want to create a ``gettext.GNUTranslations`` instance, and make the +translation functions it provides available to your templates by putting them +in the template context. + +The ``Translator`` filter needs to be added to the filters of the template +(applying it as a stream filter will likely not have the desired effect). +Furthermore it needs to be the first filter in the list, including the internal +filters that Genshi adds itself: + +.. code-block:: python + + from genshi.filters import Translator + from genshi.template import MarkupTemplate + + template = MarkupTemplate("...") + template.filters.insert(0, Translator(translations.ugettext)) + +If you're using `TemplateLoader`, you should specify a callback function in +which you add the filter: + +.. code-block:: python + + from genshi.filters import Translator + from genshi.template import TemplateLoader + + def template_loaded(template): + template.filters.insert(0, Translator(translations.ugettext)) + + loader = TemplateLoader('templates', callback=template_loaded) + template = loader.load("...") + +This approach ensures that the filter is not added everytime the template is +loaded, and thus being applied multiple times. + + +Related Considerations +====================== + +If you intend to produce an application that is fully prepared for an +international audience, there are a couple of other things to keep in mind: + +------- +Unicode +------- + +Use ``unicode`` internally, not encoded bytestrings. Only encode/decode where +data enters or exits the system. This means that your code works with characters +and not just with bytes, which is an important distinction for example when +calculating the length of a piece of text. When you need to decode/encode, it's +probably a good idea to use UTF-8. + +------------- +Date and Time +------------- + +If your application uses datetime information that should be displayed to users +in different timezones, you should try to work with UTC (universal time) +internally. Do the conversion from and to "local time" when the data enters or +exits the system. Make use the Python `datetime`_ module and the third-party +`pytz`_ package. + +-------------------------- +Formatting and Locale Data +-------------------------- + +Make sure you check out the functionality provided by the `Babel`_ project for +things like number and date formatting, locale display strings, etc. + +.. _`datetime`: http://docs.python.org/lib/module-datetime.html +.. _`pytz`: http://pytz.sourceforge.net/
--- a/doc/index.txt +++ b/doc/index.txt @@ -1,8 +1,8 @@ .. -*- mode: rst; encoding: utf-8 -*- -====== -Genshi -====== +======= +Preface +======= .. image:: logo.png :width: 225 @@ -11,20 +11,34 @@ :alt: Genshi - Generate output for the web :class: logo ---------------------------------------------------------- -Toolkit for stream-based generation of output for the web ---------------------------------------------------------- +-------------------------------------------- +Toolkit for generation of output for the web +-------------------------------------------- Genshi is a Python library that provides an integrated set of components for parsing, generating, and processing HTML, XML or other textual content for output generation on the web. The major feature is a template language, which is heavily inspired by Kid. +Installation +------------ + +* `Installing Genshi <install.html>`_ +* `Upgrading from Previous Versions <upgrade.html>`_ + +Usage +----- + * `Markup Streams <streams.html>`_ * `Templating Basics <templates.html>`_ * `XML Template Language <xml-templates.html>`_ * `Text Template Language <text-templates.html>`_ * `Using Stream Filters <filters.html>`_ * `Using XPath <xpath.html>`_ +* `Internationalization and Localization <i18n.html>`_ * `Using the Templating Plugin <plugin.html>`_ + +API Documentation +----------------- + * `Generated API Documentation <api/index.html>`_
new file mode 100644 --- /dev/null +++ b/doc/install.txt @@ -0,0 +1,88 @@ +Installing Genshi +================= + + +.. contents:: Contents + :depth: 2 +.. sectnum:: + + +Prerequisites +------------- + +* Python_ 2.4 or later +* Optional: Setuptools_ 0.6c3 or later + +.. _python: http://www.python.org/ +.. _setuptools: http://cheeseshop.python.org/pypi/setuptools + +Setuptools is only required for the `template engine plugin`_, which can be +used to integrate Genshi with Python web application frameworks such as Pylons +or TurboGears. Genshi also provides a Setuptools-based plugin that integrates +its `internationalization support`_ with the Babel_ library, but that support +can also be used without Setuptools being available (although in a slightly +less convenient fashion). + +.. _`template engine plugin`: plugin.html +.. _`internationalization support`: i18n.html +.. _babel: http://babel.edgewall.org/ + + +Installing via ``easy_install`` +------------------------------- + +If you have a recent version of Setuptools_ installed, you can directly install +Genshi using the easy_install command-line tool:: + + $ easy_install Genshi + +This downloads and installs the latest version of the Genshi package. + +If you have an older Genshi release installed and would like to upgrade, add +the ``-U`` option to the above command. + + +Installing from a Binary Installer +---------------------------------- + +Binary packages for Windows and Mac OS X are provided for Genshi. To install +from such a package, simply download and open it. + + +Installing from a Source Tarball +-------------------------------- + +Once you've downloaded and unpacked a Genshi source release, enter the +directory where the archive was unpacked, and run:: + + $ python setup.py install + +Note that you may need administrator/root privileges for this step, as this +command will by default attempt to install Genshi to the Python +``site-packages`` directory on your system. + +Genshi comes with an optional extension module written in C that is used to +improve performance in some areas. This extension is automatically compiled +when you run the ``setup.py`` script as shown above. In the case that the +extension can not be compiled, possibly due to a missing or incompatible C +compiler, the compilation is skipped. If you'd prefer Genshi to not use this +native extension module, you can explicitly bypass the compilation using the +``--without-speedups`` option:: + + $ python setup.py --without-speedups install + +For other build and installation options, please consult the easy_install_ +and/or the Python distutils_ documentation. + +.. _easy_install: http://peak.telecommunity.com/DevCenter/EasyInstall +.. _distutils: http://docs.python.org/inst/inst.html + + +Support +------- + +If you encounter any problems with Genshi, please don't hesitate to ask +questions on the Genshi `mailing list`_ or `IRC channel`_. + +.. _`mailing list`: http://genshi.edgewall.org/wiki/MailingList +.. _`irc channel`: http://genshi.edgewall.org/wiki/IrcChannel
--- a/doc/plugin.txt +++ b/doc/plugin.txt @@ -22,12 +22,11 @@ Introduction ============ -Most Python web frameworks (with the notable exception of Django_) support -a variety of different templating engines through the `Template Engine Plugin -API`_, which was first developed by the Buffet_ and TurboGears_ projects. +Some Python web frameworks support a variety of different templating engines +through the `Template Engine Plugin API`_, which was first developed by the +Buffet_ and TurboGears_ projects. .. _`Template Engine Plugin API`: http://docs.turbogears.org/1.0/TemplatePlugins -.. _`Django`: http://www.djangoproject.com/ .. _`Buffet`: http://projects.dowski.com/projects/buffet .. _`TurboGears`: http://www.turbogears.org/ @@ -72,8 +71,31 @@ format when you want to produce an Atom feed or other XML content. +Template Paths +-------------- + +How you specify template paths depends on whether you have a `search path`_ set +up or not. The search path is a list of directories that Genshi should load +templates from. Now when you request a template using a relative path such as +``mytmpl.html`` or ``foo/mytmpl.html``, Genshi will look for that file in the +directories on the search path. + +For mostly historical reasons, the Genshi template engine plugin uses a +different approach when you **haven't** configured the template search path: +you now load templates using *dotted notation*, for example ``mytmpl`` or +``foo.mytmpl``. Note how you've lost the ability to explicitly specify the +file extension: you now have to use ``.html`` for markup templates, and +``.txt`` for text templates. + +Using the search path is recommended for a number of reasons: First, it's +the native Genshi model and is thus more robust and better supported. +Second, a search path gives you much more flexibility for organizing your +application templates. And as noted above, you aren't forced to use hardcoded +filename extensions for your template files. + + Extra Implicit Objects -====================== +---------------------- The "genshi-markup" template engine plugin adds some extra functions that are made available to all templates implicitly, namely: @@ -101,16 +123,24 @@ or may not be made available by your framework. TurboGears 1.0, for example, only passes a fixed set of options to all plugins. +``genshi.allow_exec`` +-------------------------- +Whether the Python code blocks should be permitted in templates. Specify "yes" +to allow code blocks (which is the default), or "no" otherwise. Please note +that disallowing code blocks in templates does not turn Genshi into a +sandboxable template engine; there are sufficient ways to do harm even using +plain expressions. + ``genshi.auto_reload`` ---------------------- Whether the template loader should check the last modification time of template files, and automatically reload them if they have been changed. Specify "yes" to enable this reloading (which is the default), or "no" to turn it off. -.. note:: You may want to disable reloading in a production environment to gain - a slight (and possible even negligible) improvement in loading - performance, but then you'll have to manually restart the server - process anytime the templates are updated. +You probably want to disable reloading in a production environment to improve +performance of both templating loading and the processing of includes. But +remember that you'll then have to manually restart the server process anytime +the templates are updated. ``genshi.default_doctype`` -------------------------- @@ -190,6 +220,16 @@ .. _`Understanding HTML, XML and XHTML`: http://webkit.org/blog/?p=68 +``genshi.loader_callback`` +-------------------------- +The callback function that should be invoked whenever the template loader loads +a new template. + +.. note:: Unlike the other options, this option can **not** be passed as + a string value, but rather must be a reference to the actual function. + That effectively means it can not be set from (non-Python) + configuration files. + ``genshi.lookup_errors`` ------------------------ The error handling style to use in template expressions. Can be either @@ -204,6 +244,16 @@ The default value is **25**. You may want to choose a higher value if your web site uses a larger number of templates, and you have enough memory to spare. +``genshi.new_text_syntax`` +-------------------------- +Whether the new syntax for text templates should be used. Specify "yes" to +enable the new syntax, or "no" to use the old syntax. + +In the version of Genshi, the default is to use the old syntax for +backwards-compatibility, but that will change in a future release. + +.. _`search path`: + ``genshi.search_path`` ---------------------- A colon-separated list of file-system path names that the template loader should
--- a/doc/streams.txt +++ b/doc/streams.txt @@ -8,7 +8,7 @@ .. contents:: Contents - :depth: 1 + :depth: 2 .. sectnum:: @@ -22,7 +22,9 @@ * programmatically generated. For example, the functions ``XML()`` and ``HTML()`` can be used to convert -literal XML or HTML text to a markup stream:: +literal XML or HTML text to a markup stream: + +.. code-block:: pycon >>> from genshi import XML >>> stream = XML('<p class="intro">Some text and ' @@ -41,7 +43,7 @@ * ``pos`` is a ``(filename, lineno, column)`` tuple that describes where the event “comes from”. -:: +.. code-block:: pycon >>> for kind, data, pos in stream: ... print kind, `data`, pos @@ -64,7 +66,9 @@ stream, either filters that come with Genshi, or your own custom filters. A filter is simply a callable that accepts the stream as parameter, and returns -the filtered stream:: +the filtered stream: + +.. code-block:: python def noop(stream): """A filter that doesn't actually do anything with the stream.""" @@ -72,17 +76,23 @@ yield kind, data, pos Filters can be applied in a number of ways. The simplest is to just call the -filter directly:: +filter directly: + +.. code-block:: python stream = noop(stream) The ``Stream`` class also provides a ``filter()`` method, which takes an -arbitrary number of filter callables and applies them all:: +arbitrary number of filter callables and applies them all: + +.. code-block:: python stream = stream.filter(noop) Finally, filters can also be applied using the *bitwise or* operator (``|``), -which allows a syntax similar to pipes on Unix shells:: +which allows a syntax similar to pipes on Unix shells: + +.. code-block:: python stream = stream | noop @@ -90,17 +100,23 @@ ``genshi.filters``. It processes a stream of HTML markup, and strips out any potentially dangerous constructs, such as Javascript event handlers. ``HTMLSanitizer`` is not a function, but rather a class that implements -``__call__``, which means instances of the class are callable:: +``__call__``, which means instances of the class are callable: + +.. code-block:: python stream = stream | HTMLSanitizer() Both the ``filter()`` method and the pipe operator allow easy chaining of -filters:: +filters: + +.. code-block:: python from genshi.filters import HTMLSanitizer stream = stream.filter(noop, HTMLSanitizer()) -That is equivalent to:: +That is equivalent to: + +.. code-block:: python stream = stream | noop | HTMLSanitizer() @@ -116,12 +132,15 @@ events, which you'll need when you want to transmit or store the results of generating or otherwise processing markup. -The ``Stream`` class provides two methods for serialization: ``serialize()`` and -``render()``. The former is a generator that yields chunks of ``Markup`` objects -(which are basically unicode strings that are considered safe for output on the -web). The latter returns a single string, by default UTF-8 encoded. +The ``Stream`` class provides two methods for serialization: ``serialize()`` +and ``render()``. The former is a generator that yields chunks of ``Markup`` +objects (which are basically unicode strings that are considered safe for +output on the web). The latter returns a single string, by default UTF-8 +encoded. -Here's the output from ``serialize()``:: +Here's the output from ``serialize()``: + +.. code-block:: pycon >>> for output in stream.serialize(): ... print `output` @@ -135,38 +154,94 @@ <Markup u'<br/>'> <Markup u'</p>'> -And here's the output from ``render()``:: +And here's the output from ``render()``: + +.. code-block:: pycon >>> print stream.render() <p class="intro">Some text and <a href="http://example.org/">a link</a>.<br/></p> Both methods can be passed a ``method`` parameter that determines how exactly -the events are serialzed to text. This parameter can be either “xml” (the -default), “xhtml”, “html”, “text”, or a custom serializer class:: +the events are serialized to text. This parameter can be either a string or a +custom serializer class: + +.. code-block:: pycon >>> print stream.render('html') <p class="intro">Some text and <a href="http://example.org/">a link</a>.<br></p> Note how the `<br>` element isn't closed, which is the right thing to do for -HTML. +HTML. See `serialization methods`_ for more details. In addition, the ``render()`` method takes an ``encoding`` parameter, which defaults to “UTF-8”. If set to ``None``, the result will be a unicode string. The different serializer classes in ``genshi.output`` can also be used -directly:: +directly: + +.. code-block:: pycon >>> from genshi.filters import HTMLSanitizer >>> from genshi.output import TextSerializer >>> print ''.join(TextSerializer()(HTMLSanitizer()(stream))) Some text and a link. -The pipe operator allows a nicer syntax:: +The pipe operator allows a nicer syntax: + +.. code-block:: pycon >>> print stream | HTMLSanitizer() | TextSerializer() Some text and a link. +.. _`serialization methods`: + +Serialization Methods +--------------------- + +Genshi supports the use of different serialization methods to use for creating +a text representation of a markup stream. + +``xml`` + The ``XMLSerializer`` is the default serialization method and results in + proper XML output including namespace support, the XML declaration, CDATA + sections, and so on. It is not generally not suitable for serving HTML or + XHTML web pages (unless you want to use true XHTML 1.1), for which the + ``xhtml`` and ``html`` serializers described below should be preferred. + +``xhtml`` + The ``XHTMLSerializer`` is a specialization of the generic ``XMLSerializer`` + that understands the pecularities of producing XML-compliant output that can + also be parsed without problems by the HTML parsers found in modern web + browsers. Thus, the output by this serializer should be usable whether sent + as "text/html" or "application/xhtml+html" (although there are a lot of + subtle issues to pay attention to when switching between the two, in + particular with respect to differences in the DOM and CSS). + + For example, instead of rendering a script tag as ``<script/>`` (which + confuses the HTML parser in many browsers), it will produce + ``<script></script>``. Also, it will normalize any boolean attributes values + that are minimized in HTML, so that for example ``<hr noshade="1"/>`` + becomes ``<hr noshade="noshade" />``. + + This serializer supports the use of namespaces for compound documents, for + example to use inline SVG inside an XHTML document. + +``html`` + The ``HTMLSerializer`` produces proper HTML markup. The main differences + compared to ``xhtml`` serialization are that boolean attributes are + minimized, empty tags are not self-closing (so it's ``<br>`` instead of + ``<br />``), and that the contents of ``<script>`` and ``<style>`` elements + are not escaped. + +``text`` + The ``TextSerializer`` produces plain text from markup streams. This is + useful primarily for `text templates`_, but can also be used to produce + plain text output from markup templates or other sources. + +.. _`text templates`: text-templates.html + + Serialization Options --------------------- @@ -175,8 +250,8 @@ options are supported by the built-in serializers: ``strip_whitespace`` - Whether the serializer should remove trailing spaces and empty lines. Defaults - to ``True``. + Whether the serializer should remove trailing spaces and empty lines. + Defaults to ``True``. (This option is not available for serialization to plain text.) @@ -186,6 +261,38 @@ output. If provided, this declaration will override any ``DOCTYPE`` declaration in the stream. + The parameter can also be specified as a string to refer to commonly used + doctypes: + + +-----------------------------+-------------------------------------------+ + | Shorthand | DOCTYPE | + +=============================+===========================================+ + | ``html`` or | HTML 4.01 Strict | + | ``html-strict`` | | + +-----------------------------+-------------------------------------------+ + | ``html-transitional`` | HTML 4.01 Transitional | + +-----------------------------+-------------------------------------------+ + | ``html-frameset`` | HTML 4.01 Frameset | + +-----------------------------+-------------------------------------------+ + | ``html5`` | DOCTYPE proposed for the work-in-progress | + | | HTML5 standard | + +-----------------------------+-------------------------------------------+ + | ``xhtml`` or | XHTML 1.0 Strict | + | ``xhtml-strict`` | | + +-----------------------------+-------------------------------------------+ + | ``xhtml-transitional`` | XHTML 1.0 Transitional | + +-----------------------------+-------------------------------------------+ + | ``xhtml-frameset`` | XHTML 1.0 Frameset | + +-----------------------------+-------------------------------------------+ + | ``xhtml11`` | XHTML 1.1 | + +-----------------------------+-------------------------------------------+ + | ``svg`` or ``svg-full`` | SVG 1.1 | + +-----------------------------+-------------------------------------------+ + | ``svg-basic`` | SVG 1.1 Basic | + +-----------------------------+-------------------------------------------+ + | ``svg-tiny`` | SVG 1.1 Tiny | + +-----------------------------+-------------------------------------------+ + (This option is not available for serialization to plain text.) ``namespace_prefixes`` @@ -194,13 +301,28 @@ (This option is not available for serialization to HTML or plain text.) +``drop_xml_decl`` + Whether to remove the XML declaration (the ``<?xml ?>`` part at the + beginning of a document) when serializing. This defaults to ``True`` as an + XML declaration throws some older browsers into "Quirks" rendering mode. + + (This option is only available for serialization to XHTML.) + +``strip_markup`` + Whether the text serializer should detect and remove any tags or entity + encoded characters in the text. + + (This option is only available for serialization to plain text.) + Using XPath =========== XPath can be used to extract a specific subset of the stream via the -``select()`` method:: +``select()`` method: + +.. code-block:: pycon >>> substream = stream.select('a') >>> substream @@ -211,7 +333,9 @@ Often, streams cannot be reused: in the above example, the sub-stream is based on a generator. Once it has been serialized, it will have been fully consumed, and cannot be rendered again. To work around this, you can wrap such a stream -in a ``list``:: +in a ``list``: + +.. code-block:: pycon >>> from genshi import Stream >>> substream = Stream(list(stream.select('a'))) @@ -251,16 +375,20 @@ ``(tagname, attrs)``, where ``tagname`` is a ``QName`` instance describing the qualified name of the tag, and ``attrs`` is an ``Attrs`` instance containing the attribute names and values associated with the tag (excluding namespace -declarations):: +declarations): - START, (QName(u'p'), Attrs([(u'class', u'intro')])), pos +.. code-block:: python + + START, (QName(u'p'), Attrs([(QName(u'class'), u'intro')])), pos END --- The closing tag of an element. The ``data`` item of end events consists of just a ``QName`` instance -describing the qualified name of the tag:: +describing the qualified name of the tag: + +.. code-block:: python END, QName(u'p'), pos @@ -268,7 +396,9 @@ ---- Character data outside of elements and comments. -For text events, the ``data`` item should be a unicode object:: +For text events, the ``data`` item should be a unicode object: + +.. code-block:: python TEXT, u'Hello, world!', pos @@ -279,7 +409,9 @@ The ``data`` item of this kind of event is a tuple of the form ``(prefix, uri)``, where ``prefix`` is the namespace prefix and ``uri`` is the full URI to which the prefix is bound. Both should be unicode objects. If the -namespace is not bound to any prefix, the ``prefix`` item is an empty string:: +namespace is not bound to any prefix, the ``prefix`` item is an empty string: + +.. code-block:: python START_NS, (u'svg', u'http://www.w3.org/2000/svg'), pos @@ -288,7 +420,9 @@ The end of a namespace mapping. The ``data`` item of such events consists of only the namespace prefix (a -unicode object):: +unicode object): + +.. code-block:: python END_NS, u'svg', pos @@ -299,7 +433,9 @@ For this type of event, the ``data`` item is a tuple of the form ``(name, pubid, sysid)``, where ``name`` is the name of the root element, ``pubid`` is the public identifier of the DTD (or ``None``), and ``sysid`` is -the system identifier of the DTD (or ``None``):: +the system identifier of the DTD (or ``None``): + +.. code-block:: python DOCTYPE, (u'html', u'-//W3C//DTD XHTML 1.0 Transitional//EN', \ u'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'), pos @@ -309,7 +445,9 @@ A comment. For such events, the ``data`` item is a unicode object containing all character -data between the comment delimiters:: +data between the comment delimiters: + +.. code-block:: python COMMENT, u'Commented out', pos @@ -320,7 +458,9 @@ The ``data`` item is a tuple of the form ``(target, data)`` for processing instructions, where ``target`` is the target of the PI (used to identify the application by which the instruction should be processed), and ``data`` is text -following the target (excluding the terminating question mark):: +following the target (excluding the terminating question mark): + +.. code-block:: python PI, (u'php', u'echo "Yo" '), pos @@ -328,7 +468,9 @@ ----------- Marks the beginning of a ``CDATA`` section. -The ``data`` item for such events is always ``None``:: +The ``data`` item for such events is always ``None``: + +.. code-block:: python START_CDATA, None, pos @@ -336,6 +478,8 @@ --------- Marks the end of a ``CDATA`` section. -The ``data`` item for such events is always ``None``:: +The ``data`` item for such events is always ``None``: + +.. code-block:: python END_CDATA, None, pos
deleted file mode 100644 index 90e92682135d3f7213332f870f973bd06d6d57ee..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch literal 0 Hc$@<O00001
deleted file mode 100644 --- a/doc/style/docutils.css +++ /dev/null @@ -1,277 +0,0 @@ -/* -:Author: David Goodger -:Contact: goodger@users.sourceforge.net -:Date: $Date: 2005-12-18 01:56:14 +0100 (Sun, 18 Dec 2005) $ -:Revision: $Revision: 4224 $ -:Copyright: This stylesheet has been placed in the public domain. - -Default cascading style sheet for the HTML output of Docutils. - -See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to -customize this style sheet. -*/ - -/* used to remove borders from tables and images */ -.borderless, table.borderless td, table.borderless th { - border: 0 } - -table.borderless td, table.borderless th { - /* Override padding for "table.docutils td" with "! important". - The right padding separates the table cells. */ - padding: 0 0.5em 0 0 ! important } - -.first { - /* Override more specific margin styles with "! important". */ - margin-top: 0 ! important } - -.last, .with-subtitle { - margin-bottom: 0 ! important } - -.hidden { - display: none } - -a.toc-backref { - text-decoration: none ; - color: black } - -blockquote.epigraph { - margin: 2em 5em ; } - -dl.docutils dd { - margin-bottom: 0.5em } - -dl.docutils dt { - font-weight: bold } - -div.abstract { - margin: 2em 5em } - -div.abstract p.topic-title { - font-weight: bold ; - text-align: center } - -div.admonition, div.attention, div.caution, div.danger, div.error, -div.hint, div.important, div.note, div.tip, div.warning { - margin: 2em ; - border: medium outset ; - padding: 1em } - -div.admonition p.admonition-title, div.hint p.admonition-title, -div.important p.admonition-title, div.note p.admonition-title, -div.tip p.admonition-title { - font-weight: bold ; - font-family: sans-serif } - -div.attention p.admonition-title, div.caution p.admonition-title, -div.danger p.admonition-title, div.error p.admonition-title, -div.warning p.admonition-title { - color: red ; - font-weight: bold ; - font-family: sans-serif } - -/* Uncomment (and remove this text!) to get reduced vertical space in - compound paragraphs. -div.compound .compound-first, div.compound .compound-middle { - margin-bottom: 0.5em } - -div.compound .compound-last, div.compound .compound-middle { - margin-top: 0.5em } -*/ - -div.dedication { - margin: 2em 5em ; - text-align: center ; - font-style: italic } - -div.dedication p.topic-title { - font-weight: bold ; - font-style: normal } - -div.figure { - margin-left: 2em ; - margin-right: 2em } - -div.footer, div.header { - clear: both; - font-size: smaller } - -div.line-block { - display: block ; - margin-top: 1em ; - margin-bottom: 1em } - -div.line-block div.line-block { - margin-top: 0 ; - margin-bottom: 0 ; - margin-left: 1.5em } - -div.sidebar { - margin-left: 1em ; - border: medium outset ; - padding: 1em ; - background-color: #ffffee ; - width: 40% ; - float: right ; - clear: right } - -div.sidebar p.rubric { - font-family: sans-serif ; - font-size: medium } - -div.system-messages { - margin: 5em } - -div.system-messages h1 { - color: red } - -div.system-message { - border: medium outset ; - padding: 1em } - -div.system-message p.system-message-title { - color: red ; - font-weight: bold } - -div.topic { - margin: 2em } - -h1.section-subtitle, h2.section-subtitle, h3.section-subtitle, -h4.section-subtitle, h5.section-subtitle, h6.section-subtitle { - margin-top: 0.4em } - -h1.title { - text-align: center } - -h2.subtitle { - text-align: center } - -hr.docutils { - width: 75% } - -img.align-left { - clear: left } - -img.align-right { - clear: right } - -ol.simple, ul.simple { - margin-bottom: 1em } - -ol.arabic { - list-style: decimal } - -ol.loweralpha { - list-style: lower-alpha } - -ol.upperalpha { - list-style: upper-alpha } - -ol.lowerroman { - list-style: lower-roman } - -ol.upperroman { - list-style: upper-roman } - -p.attribution { - text-align: right ; - margin-left: 50% } - -p.caption { - font-style: italic } - -p.credits { - font-style: italic ; - font-size: smaller } - -p.label { - white-space: nowrap } - -p.rubric { - font-weight: bold ; - font-size: larger ; - color: maroon ; - text-align: center } - -p.sidebar-title { - font-family: sans-serif ; - font-weight: bold ; - font-size: larger } - -p.sidebar-subtitle { - font-family: sans-serif ; - font-weight: bold } - -p.topic-title { - font-weight: bold } - -pre.address { - margin-bottom: 0 ; - margin-top: 0 ; - font-family: serif ; - font-size: 100% } - -pre.literal-block, pre.doctest-block { - margin-left: 2em ; - margin-right: 2em ; - background-color: #eeeeee } - -span.classifier { - font-family: sans-serif ; - font-style: oblique } - -span.classifier-delimiter { - font-family: sans-serif ; - font-weight: bold } - -span.interpreted { - font-family: sans-serif } - -span.option { - white-space: nowrap } - -span.pre { - white-space: pre } - -span.problematic { - color: red } - -span.section-subtitle { - /* font-size relative to parent (h1..h6 element) */ - font-size: 80% } - -table.citation { - border-left: solid 1px gray; - margin-left: 1px } - -table.docinfo { - margin: 2em 4em } - -table.docutils { - margin-top: 0.5em ; - margin-bottom: 0.5em } - -table.footnote { - border-left: solid 1px black; - margin-left: 1px } - -table.docutils td, table.docutils th, -table.docinfo td, table.docinfo th { - padding-left: 0.5em ; - padding-right: 0.5em ; - vertical-align: top } - -table.docutils th.field-name, table.docinfo th.docinfo-name { - font-weight: bold ; - text-align: left ; - white-space: nowrap ; - padding-left: 0 } - -h1 tt.docutils, h2 tt.docutils, h3 tt.docutils, -h4 tt.docutils, h5 tt.docutils, h6 tt.docutils { - font-size: 100% } - -tt.docutils { - background-color: #eeeeee } - -ul.auto-toc { - list-style-type: none }
deleted file mode 100644 --- a/doc/style/edgewall.css +++ /dev/null @@ -1,67 +0,0 @@ -@import(docutils.css); - -html, body { height: 100%; margin: 0; padding: 0; } -html, body { background: #4b4d4d url(bkgnd_pattern.png); color: #000; } -body, th, td { - font: normal small Verdana,Arial,'Bitstream Vera Sans',Helvetica,sans-serif; -} -pre, code, tt { font-size: medium; } -h1, h2, h3, h4 { - border-bottom: 1px solid #ccc; - font-family: Arial,Verdana,'Bitstream Vera Sans',Helvetica,sans-serif; - font-weight: bold; letter-spacing: -0.018em; -} -h1 { font-size: 19px; margin: 2em 0 .5em; } -h2 { font-size: 16px; margin: 1.5em 0 .5em; } -h3 { font-size: 14px; margin: 1.2em 0 .5em; } -hr { border: none; border-top: 1px solid #ccb; margin: 2em 0; } -p { margin: 0 0 1em; } - -table { border: 1px solid #999; border-width: 0 1px 0 0; - border-collapse: separate; border-spacing: 0; -} -table thead th { background: #999; border: 1px solid #999;; color: #fff; - font-weight: bold; -} -table td { border: 1px solid #ccc; border-width: 0 0 1px 1px; padding: .3em; } - -:link, :visited { text-decoration: none; border-bottom: 1px dotted #bbb; - color: #b00; -} -:link:hover, :visited:hover { background-color: #eee; color: #555; } -:link img, :visited img { border: none } -h1 :link, h1 :visited ,h2 :link, h2 :visited, h3 :link, h3 :visited, -h4 :link, h4 :visited, h5 :link, h5 :visited, h6 :link, h6 :visited { - color: #000; -} - -div.document { background: #fff url(shadow.gif) right top repeat-y; - border-left: 1px solid #000; margin: 0 auto 0 40px; - min-height: 100%; width: 54em; padding: 0 180px 1px 20px; -} -h1.title, div.document#genshi h1 { border: none; color: #666; - font-size: x-large; margin: 0 -20px 1em; padding: 2em 20px 0; -} -h1.title { background: url(vertbars.png) repeat-x; } -div.document#genshi h1.title { text-indent: -4000px; } -div.document#genshi h1 { text-align: center; } -pre.literal-block { background: #d7d7d7; border: 1px solid #e6e6e6; color: #000; - margin: 1em 1em; padding: .25em; overflow: auto; -} - -div.contents { font-size: 90%; position: absolute; position: fixed; - margin-left: 80px; left: 60em; top: 30px; right: 0; -} -div.contents .topic-title { display: none; } -div.contents ul { list-style: none; padding-left: 0; } -div.contents :link, div.contents :visited { color: #c6c6c6; border: none; - display: block; padding: 3px 5px 3px 10px; -} -div.contents :link:hover, div.contents :visited:hover { background: #000; - color: #fff; -} - -p.admonition-title { font-weight: bold; margin-bottom: 0; } -div.note, div.warning { font-style: italic; margin-left: 2em; - margin-right: 2em; -}
deleted file mode 100644 --- a/doc/style/epydoc.css +++ /dev/null @@ -1,136 +0,0 @@ -html { background: #4b4d4d url(../style/bkgnd_pattern.png); margin: 0; - padding: 1em 1em 3em; -} -body { background: #fff url(../style/vertbars.png) repeat-x; - border: 1px solid #000; color: #000; margin: 1em 0; padding: 0 1em 1em; -} -body, th, td { - font: normal small Verdana,Arial,'Bitstream Vera Sans',Helvetica,sans-serif; -} -h1, h2, h3, h4 { - font-family: Arial,Verdana,'Bitstream Vera Sans',Helvetica,sans-serif; - font-weight: bold; letter-spacing: -0.018em; -} -h1 { font-size: 19px; margin: 2em 0 .5em; } -h2 { font-size: 16px; margin: 1.5em 0 .5em; } -h3 { font-size: 14px; margin: 1.2em 0 .5em; } -hr { border: none; border-top: 1px solid #ccb; margin: 2em 0; } -p { margin: 0 0 1em; } -:link, :visited { text-decoration: none; border-bottom: 1px dotted #bbb; - color: #b00; -} -:link:hover, :visited:hover { background-color: #eee; color: #555; } - -table { border: none; border-collapse: collapse; } - -table.navbar { background: #000; color: #fff; margin: 2em 0 .33em; } -table.navbar th { border: 1px solid #000; font-weight: bold; padding: 1px; } -table.navbar :link, table.navbar :visited { border: none; color: #fff; } -table.navbar :link:hover, table.navbar :visited:hover { background: none; - text-decoration: underline overline; -} -table.navbar th.navbar-select { background: #fff; color: #000; } -span.breadcrumbs { color: #666; font-size: 95%; } -h1.epydoc { border: none; color: #666; - font-size: x-large; margin: 1em 0 0; padding: 0; -} -pre.base-tree { color: #666; margin: 0; padding: 0; } -pre.base-tree :link, pre.base-tree :visited { border: none; } -pre.py-doctest, pre.variable, pre.rst-literal-block { background: #eee; - border: 1px solid #e6e6e6; color: #000; margin: 1em; padding: .25em; - overflow: auto; -} -pre.variable { margin: 0; } - -/* Summary tables */ - -table.summary { margin: .5em 0; } -table.summary tr.table-header { background: #f7f7f0; } -table.summary td.table-header { color: #666; font-weight: bold; } -table.summary th.group-header { background: #f7f7f0; color: #666; - font-size: 90%; font-weight: bold; text-align: left; -} -table.summary th, table.summary td { border: 1px solid #d7d7d7; } -table.summary th th, table.summary td td { border: none; } -table.summary td.summary table td { color: #666; font-size: 90%; } -table.summary td.summary table br { display: none; } -table.summary td.summary span.summary-type { font-family: monospace; - font-size: 90%; -} -table.summary td.summary span.summary-type code { font-size: 110%; } -p.indent-wrapped-lines { color: #999; font-size: 85%; margin: 0; - padding: 0 0 0 7em; text-indent: -7em; -} -p.indent-wrapped-lines code { color: #999; font-size: 115%; } -p.indent-wrapped-lines :link, p.indent-wrapped-lines :visited { border: none; } -.summary-sig { display: block; font-family: monospace; font-size: 120%; - margin-bottom: .5em; -} -.summary-sig-name { font-weight: bold; } -.summary-sig-arg { color: #333; } -.summary-sig :link, .summary-sig :visited { border: none; } -.summary-name { font-family: monospace; font-weight: bold; } - -/* Details tables */ - -table.details { margin: 2em 0 0; } -div table.details { margin-top: 0; } -table.details tr.table-header { background: transparent; } -table.details td.table-header { border-bottom: 1px solid #ccc; padding: 2em 0 0; } -table.details span.table-header { - font: bold 140% Arial,Verdana,'Bitstream Vera Sans',Helvetica,sans-serif; - letter-spacing: -0.018em; -} -table.details th, table.details td { border: none; } -table.details th th, table.details td td { border: none; } -table.details td { padding-left: 2em; } -table.details td td { padding-left: 0; } -table.details h3.epydoc { margin-left: -2em; } -table.details h3.epydoc .sig { color: #999; font-family: monospace; } -table.details h3.epydoc .sig-name { color: #000; } -table.details h3.epydoc .sig-arg { color: #666; } -table.details h3.epydoc .sig-default { font-size: 95%; font-weight: normal; } -table.details h3.epydoc .sig-default code { font-weight: normal; } -table.details h3.epydoc .fname { color: #999; font-size: 90%; - font-style: italic; font-weight: normal; line-height: 1.6em; -} - -dl dt { color: #666; margin-top: 1em; } -dl dd { margin: 0; padding-left: 2em; } -dl.fields { margin: 1em 0; padding: 0; } -dl.fields dt { color: #666; margin-top: 1em; } -dl.fields dd ul { margin: 0; padding: 0; } -div.fields { font-size: 90%; margin: 0 0 2em 2em; } -div.fields p { margin-bottom: 0.5em; } - -table td.footer { color: #999; font-size: 85%; margin-top: 3em; - padding: 0 3em 1em; position: absolute; width: 80%; } -table td.footer :link, table td.footer :visited { border: none; color: #999; } -table td.footer :link:hover, table td.footer :visited:hover { - background: transparent; text-decoration: underline; -} - -/* Syntax highlighting */ - -.py-prompt, .py-more, .variable-ellipsis, .variable-op { color: #999; } -.variable-group { color: #666; font-weight: bold; } -.py-string, .variable-string, .variable-quote { color: #093; } -.py-comment { color: #06f; font-style: italic; } -.py-keyword { color: #00f; } -.py-output { background: #f6f6f0; color: #666; font-weight: bold; } - -/* Index */ - -table.link-index { background: #f6f6f0; border: none; margin-top: 1em; } -table.link-index td.link-index { border: none; font-family: monospace; - font-weight: bold; padding: .5em 1em; -} -table.link-index td table, table.link-index td td { border: none; } -table.link-index .index-where { color: #999; - font-family: Verdana,Arial,'Bitstream Vera Sans',Helvetica,sans-serif; - font-size: 90%; font-weight: normal; line-height: 1.6em; -} -table.link-index .index-where :link, table.link-index .index-where :visited { - border: none; color: #666; -} -h2.epydoc { color: #999; font-size: 200%; line-height: 10px; }
deleted file mode 100644 index 326cd1b37dcd637d2e8b7193fe0011f51cca3ce9..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch literal 0 Hc$@<O00001
deleted file mode 100644 index 42ae3f86d7f1513fd585ad02959afbb37ad476cb..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch literal 0 Hc$@<O00001
--- a/doc/templates.txt +++ b/doc/templates.txt @@ -32,7 +32,9 @@ used to generate any kind of HTML or XML output, as they provide many advantages over simple text-based templates (such as automatic escaping of strings). -The following illustrates a very basic Genshi markup template:: +The following illustrates a very basic Genshi markup template: + +.. code-block:: genshi <?python title = "A Genshi Template" @@ -60,7 +62,9 @@ (c) usage of templates directives (``py:content`` and ``py:for``) (d) an inline Python expression (``${fruit}``). -The template would generate output similar to this:: +The template would generate output similar to this: + +.. code-block:: genshi <html> <head> @@ -79,7 +83,9 @@ A *text template* is a simple plain text document that can also contain embedded Python code. Text templates can be used to generate simple *non-markup* text -formats, such as the body of an plain text email. For example:: +formats, such as the body of an plain text email. For example: + +.. code-block:: genshitext Dear $name, @@ -103,33 +109,44 @@ be made available to the template as keyword arguments. * Serialize the resulting stream using its ``render()`` method. -For example:: +For example: + +.. code-block:: pycon >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('<h1>Hello, $name!</h1>') >>> stream = tmpl.generate(name='world') - >>> print stream.render() + >>> print stream.render('xhtml') <h1>Hello, world!</h1> -Using a text template is similar:: +.. note:: See the Serialization_ section of the `Markup Streams`_ page for + information on configuring template output options. + +Using a text template is similar: + +.. code-block:: pycon >>> from genshi.template import TextTemplate >>> tmpl = TextTemplate('Hello, $name!') >>> stream = tmpl.generate(name='world') - >>> print stream.render() + >>> print stream Hello, world! -.. note:: See the Serialization_ section of the `Markup Streams`_ page for - information on configuring template output options. +.. note:: If you want to use text templates, you should consider using the + ``NewTextTemplate`` class instead of simply ``TextTemplate``. See + the `Text Template Language`_ page. .. _serialization: streams.html#serialization +.. _`Text Template Language`: text-templates.html .. _`Markup Streams`: streams.html Using a template loader provides the advantage that “compiled” templates are automatically cached, and only parsed again when the template file changes. In addition, it enables the use of a *template search path*, allowing template directories to be spread across different file-system locations. Using a -template loader would generally look as follows:: +template loader would generally look as follows: + +.. code-block:: python from genshi.template import TemplateLoader loader = TemplateLoader([templates_dir1, templates_dir2]) @@ -158,7 +175,9 @@ If the expression starts with a letter and contains only letters, digits, dots, and underscores, the curly braces may be omitted. In all other cases, the braces are required so that the template processor knows where the expression -ends:: +ends: + +.. code-block:: pycon >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('<em>${items[0].capitalize()} item</em>') @@ -168,7 +187,9 @@ Expressions support the full power of Python. In addition, it is possible to access items in a dictionary using “dotted notation” (i.e. as if they were attributes), and vice-versa (i.e. access attributes as if they were items in a -dictionary):: +dictionary): + +.. code-block:: pycon >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('<em>${dict.foo}</em>') @@ -182,31 +203,90 @@ See `Error Handling`_ below for details on how such errors are handled. +Escaping +======== + +If you need to include a literal dollar sign in the output where Genshi would +normally detect an expression, you can simply add another dollar sign: + +.. code-block:: pycon + + >>> from genshi.template import MarkupTemplate + >>> tmpl = MarkupTemplate('<em>$foo</em>') # Wanted "$foo" as literal output + >>> print tmpl.generate() + Traceback (most recent call last): + ... + UndefinedError: "foo" not defined + >>> tmpl = MarkupTemplate('<em>$$foo</em>') + >>> print tmpl.generate() + <em>$foo</em> + +But note that this is not necessary if the characters following the dollar sign +do not qualify as an expression. For example, the following needs no escaping: + +.. code-block:: pycon + + >>> tmpl = MarkupTemplate('<script>$(function() {})</script>') + >>> print tmpl.generate() + <script>$(function() {})</script> + +On the other hand, Genshi will always replace two dollar signs in text with a +single dollar sign, so you'll need to use three dollar signs to get two in the +output: + +.. code-block:: pycon + + >>> tmpl = MarkupTemplate('<script>$$$("div")</script>') + >>> print tmpl.generate() + <script>$$("div")</script> + + .. _`code blocks`: Code Blocks =========== -XML templates also support full Python code blocks using the ``<?python ?>`` -processing instruction:: +Templates also support full Python code blocks, using the ``<?python ?>`` +processing instruction in XML templates: + +.. code-block:: genshi <div> <?python from genshi.builder import tag def greeting(name): - return tag.b('Hello, %s!' % name') ?> + return tag.b('Hello, %s!' % name) ?> ${greeting('world')} </div> -This will produce the following output:: +This will produce the following output: + +.. code-block:: xml <div> <b>Hello, world!</b> </div> +In text templates (although only those using the new syntax introduced in +Genshi 0.5), code blocks use the special ``{% python %}`` directive: + +.. code-block:: genshitext + + {% python + from genshi.builder import tag + def greeting(name): + return 'Hello, %s!' % name + %} + ${greeting('world')} + +This will produce the following output:: + + Hello, world! + + Code blocks can import modules, define classes and functions, and basically do anything you can do in normal Python code. What code blocks can *not* do is to -produce content that is included directly in the generated page. +produce content that is emitted directly tp the generated output. .. note:: Using the ``print`` statement will print to the standard output stream, just as it does for other Python code in your application. @@ -220,7 +300,13 @@ design. If you're using many code blocks, that may be a sign that you should move such code into separate Python modules. -.. note:: Code blocks are not currently supported in text templates. +If you'd rather not allow the use of Python code blocks in templates, you can +simply set the ``allow_exec`` parameter (available on the ``Template`` and the +``TemplateLoader`` initializers) to ``False``. In that case Genshi will raise +a syntax error when a ``<?python ?>`` processing instruction is encountered. +But please note that disallowing code blocks in templates does not turn Genshi +into a sandboxable template engine; there are sufficient ways to do harm even +using plain expressions. .. _`error handling`: @@ -228,62 +314,85 @@ Error Handling ============== -By default, Genshi allows you to access variables that are not defined, without -raising a ``NameError`` exception as regular Python code would:: +By default, Genshi raises an ``UndefinedError`` if a template expression +attempts to access a variable that is not defined: + +.. code-block:: pycon >>> from genshi.template import MarkupTemplate >>> tmpl = MarkupTemplate('<p>${doh}</p>') + >>> tmpl.generate().render('xhtml') + Traceback (most recent call last): + ... + UndefinedError: "doh" not defined + +You can change this behavior by setting the variable lookup mode to "lenient". +In that case, accessing undefined variables returns an `Undefined` object, +meaning that the expression does not fail immediately. See below for details. + +If you need to check whether a variable exists in the template context, use the +defined_ or the value_of_ function described below. To check for existence of +attributes on an object, or keys in a dictionary, use the ``hasattr()``, +``getattr()`` or ``get()`` functions, or the ``in`` operator, just as you would +in regular Python code: + + >>> from genshi.template import MarkupTemplate + >>> tmpl = MarkupTemplate('<p>${defined("doh")}</p>') + >>> print tmpl.generate().render('xhtml') + <p>False</p> + +.. note:: Lenient error handling was the default in Genshi prior to version 0.5. + Strict mode was introduced in version 0.4, and became the default in + 0.5. The reason for this change was that the lenient error handling + was masking actual errors in templates, thereby also making it harder + to debug some problems. + + +.. _`lenient`: + +Lenient Mode +------------ + +If you instruct Genshi to use the lenient variable lookup mode, it allows you +to access variables that are not defined, without raising an ``UndefinedError``. + +This mode can be chosen by passing the ``lookup='lenient'`` keyword argument to +the template initializer, or by passing the ``variable_lookup='lenient'`` +keyword argument to the ``TemplateLoader`` initializer: + +.. code-block:: pycon + + >>> from genshi.template import MarkupTemplate + >>> tmpl = MarkupTemplate('<p>${doh}</p>', lookup='lenient') >>> print tmpl.generate().render('xhtml') <p></p> You *will* however get an exception if you try to call an undefined variable, or -do anything else with it, such as accessing its attributes:: +do anything else with it, such as accessing its attributes: + +.. code-block:: pycon >>> from genshi.template import MarkupTemplate - >>> tmpl = MarkupTemplate('<p>${doh.oops}</p>') + >>> tmpl = MarkupTemplate('<p>${doh.oops}</p>', lookup='lenient') >>> print tmpl.generate().render('xhtml') Traceback (most recent call last): ... UndefinedError: "doh" not defined If you need to know whether a variable is defined, you can check its type -against the ``Undefined`` class, for example in a conditional directive:: +against the ``Undefined`` class, for example in a conditional directive: + +.. code-block:: pycon >>> from genshi.template import MarkupTemplate - >>> tmpl = MarkupTemplate('<p>${type(doh) is not Undefined}</p>') + >>> tmpl = MarkupTemplate('<p>${type(doh) is not Undefined}</p>', + ... lookup='lenient') >>> print tmpl.generate().render('xhtml') <p>False</p> Alternatively, the built-in functions defined_ or value_of_ can be used in this case. -Strict Mode ------------ - -In addition to the default "lenient" error handling, Genshi lets you use a less -forgiving mode if you prefer errors blowing up loudly instead of being ignored -silently. - -This mode can be chosen by passing the ``lookup='strict'`` keyword argument to -the template initializer, or by passing the ``variable_lookup='strict'`` keyword -argument to the ``TemplateLoader`` initializer:: - - >>> from genshi.template import MarkupTemplate - >>> tmpl = MarkupTemplate('<p>${doh}</p>', lookup='strict') - >>> print tmpl.generate().render('xhtml') - Traceback (most recent call last): - ... - UndefinedError: "doh" not defined - -When using strict mode, any reference to an undefined variable, as well as -trying to access an non-existing item or attribute of an object, will cause an -``UndefinedError`` to be raised immediately. - -.. note:: While this mode is currently not the default, it may be promoted to - the default in future versions of Genshi. In general, the default - lenient error handling mode can be considered dangerous as it silently - ignores typos. - Custom Modes ------------
--- a/doc/text-templates.txt +++ b/doc/text-templates.txt @@ -6,10 +6,7 @@ In addition to the XML-based template language, Genshi provides a simple text-based template language, intended for basic plain text generation needs. -The language is similar to Cheetah_ or Velocity_. - -.. _cheetah: http://cheetahtemplate.org/ -.. _velocity: http://jakarta.apache.org/velocity/ +The language is similar to the Django_ template language. This document describes the template language and will be most useful as reference to those developing Genshi text templates. Templates are text files of @@ -20,6 +17,13 @@ See `Genshi Templating Basics <templates.html>`_ for general information on embedding Python code in templates. +.. note:: Actually, Genshi currently has two different syntaxes for text + templates languages: One implemented by the class ``OldTextTemplate`` + and another implemented by ``NewTextTemplate``. This documentation + concentrates on the latter, which is planned to completely replace the + older syntax. The older syntax is briefly described under legacy_. + +.. _django: http://www.djangoproject.com/ .. contents:: Contents :depth: 3 @@ -32,36 +36,35 @@ Template Directives ------------------- -Directives are lines starting with a ``#`` character followed immediately by -the directive name. They can affect how the template is rendered in a number of -ways: Genshi provides directives for conditionals and looping, among others. +Directives are template commands enclosed by ``{% ... %}`` characters. They can +affect how the template is rendered in a number of ways: Genshi provides +directives for conditionals and looping, among others. -Directives must be on separate lines, and the ``#`` character must be be the -first non-whitespace character on that line. Each directive must be “closed” -using a ``#end`` marker. You can add after the ``#end`` marker, for example to -document which directive is being closed, or even the expression associated with -that directive. Any text after ``#end`` (but on the same line) is ignored, -and effectively treated as a comment. +Each directive must be terminated using an ``{% end %}`` marker. You can add +a string inside the ``{% end %}`` marker, for example to document which +directive is being closed, or even the expression associated with that +directive. Any text after ``end`` inside the delimiters is ignored, and +effectively treated as a comment. -If you want to include a literal ``#`` in the output, you need to escape it -by prepending a backslash character (``\``). Note that this is **not** required -if the ``#`` isn't immediately followed by a letter, or it isn't the first -non-whitespace character on the line. +If you want to include a literal delimiter in the output, you need to escape it +by prepending a backslash character (``\``). Conditional Sections ==================== -.. _`#if`: - -``#if`` ---------- +.. _`if`: -The content is only rendered if the expression evaluates to a truth value:: +``{% if %}`` +------------ - #if foo +The content is only rendered if the expression evaluates to a truth value: + +.. code-block:: genshitext + + {% if foo %} ${bar} - #end + {% end %} Given the data ``foo=True`` and ``bar='Hello'`` in the template context, this would produce:: @@ -69,54 +72,46 @@ Hello -.. _`#choose`: -.. _`#when`: -.. _`#otherwise`: - -``#choose`` -------------- +.. _`choose`: +.. _`when`: +.. _`otherwise`: -The ``#choose`` directive, in combination with the directives ``#when`` and -``#otherwise`` provides advanced contional processing for rendering one of -several alternatives. The first matching ``#when`` branch is rendered, or, if -no ``#when`` branch matches, the ``#otherwise`` branch is be rendered. +``{% choose %}`` +---------------- -If the ``#choose`` directive has no argument the nested ``#when`` directives -will be tested for truth:: +The ``choose`` directive, in combination with the directives ``when`` and +``otherwise``, provides advanced contional processing for rendering one of +several alternatives. The first matching ``when`` branch is rendered, or, if +no ``when`` branch matches, the ``otherwise`` branch is be rendered. + +If the ``choose`` directive has no argument the nested ``when`` directives will +be tested for truth: + +.. code-block:: genshitext The answer is: - #choose - #when 0 == 1 - 0 - #end - #when 1 == 1 - 1 - #end - #otherwise - 2 - #end - #end + {% choose %} + {% when 0 == 1 %}0{% end %} + {% when 1 == 1 %}1{% end %} + {% otherwise %}2{% end %} + {% end %} This would produce the following output:: The answer is: - 1 + 1 -If the ``#choose`` does have an argument, the nested ``#when`` directives will -be tested for equality to the parent ``#choose`` value:: +If the ``choose`` does have an argument, the nested ``when`` directives will +be tested for equality to the parent ``choose`` value: + +.. code-block:: genshitext The answer is: - #choose 1 - #when 0 - 0 - #end - #when 1 - 1 - #end - #otherwise - 2 - #end - #end + {% choose 1 %}\ + {% when 0 %}0{% end %}\ + {% when 1 %}1{% end %}\ + {% otherwise %}2{% end %}\ + {% end %} This would produce the following output:: @@ -127,17 +122,19 @@ Looping ======= -.. _`#for`: +.. _`for`: -``#for`` ----------- +``{% for %}`` +------------- -The content is repeated for every item in an iterable:: +The content is repeated for every item in an iterable: + +.. code-block:: genshitext Your items: - #for item in items + {% for item in items %}\ * ${item} - #end + {% end %} Given ``items=[1, 2, 3]`` in the context data, this would produce:: @@ -150,19 +147,21 @@ Snippet Reuse ============= -.. _`#def`: +.. _`def`: .. _`macros`: -``#def`` ----------- +``{% def %}`` +------------- -The ``#def`` directive can be used to create macros, i.e. snippets of template +The ``def`` directive can be used to create macros, i.e. snippets of template text that have a name and optionally some parameters, and that can be inserted -in other places:: +in other places: - #def greeting(name) +.. code-block:: genshitext + + {% def greeting(name) %} Hello, ${name}! - #end + {% end %} ${greeting('world')} ${greeting('everyone else')} @@ -171,13 +170,15 @@ Hello, world! Hello, everyone else! -If a macro doesn't require parameters, it can be defined as well as called -without the parenthesis. For example:: +If a macro doesn't require parameters, it can be defined without the +parenthesis. For example: - #def greeting +.. code-block:: genshitext + + {% def greeting %} Hello, world! - #end - ${greeting} + {% end %} + ${greeting()} The above would be rendered to:: @@ -185,15 +186,17 @@ .. _includes: -.. _`#include`: +.. _`include`: -``#include`` ------------- +``{% include %}`` +----------------- To reuse common parts of template text across template files, you can include -other files using the ``#include`` directive:: +other files using the ``include`` directive: - #include "base.txt" +.. code-block:: genshitext + + {% include base.txt %} Any content included this way is inserted into the generated output. The included template sees the context data as it exists at the point of the @@ -206,11 +209,13 @@ the included file at "``myapp/base.txt``". You can also use Unix-style relative paths, for example "``../base.txt``" to look in the parent directory. -Just like other directives, the argument to the ``#include`` directive accepts +Just like other directives, the argument to the ``include`` directive accepts any Python expression, so the path to the included template can be determined -dynamically:: +dynamically: - #include '%s.txt' % filename +.. code-block:: genshitext + + {% include ${'%s.txt' % filename} %} Note that a ``TemplateNotFound`` exception is raised if an included file can't be found. @@ -221,23 +226,25 @@ Variable Binding ================ -.. _`#with`: +.. _`with`: -``#with`` ------------ +``{% with %}`` +-------------- -The ``#with`` directive lets you assign expressions to variables, which can +The ``{% with %}`` directive lets you assign expressions to variables, which can be used to make expressions inside the directive less verbose and more efficient. For example, if you need use the expression ``author.posts`` more than once, and that actually results in a database query, assigning the results to a variable using this directive would probably help. -For example:: +For example: + +.. code-block:: genshitext Magic numbers! - #with y=7; z=x+10 + {% with y=7; z=x+10 %} $x $y $z - #end + {% end %} Given ``x=42`` in the context data, this would produce:: @@ -245,17 +252,109 @@ 42 7 52 Note that if a variable of the same name already existed outside of the scope -of the ``#with`` directive, it will **not** be overwritten. Instead, it will -have the same value it had prior to the ``#with`` assignment. Effectively, +of the ``with`` directive, it will **not** be overwritten. Instead, it will +have the same value it had prior to the ``with`` assignment. Effectively, this means that variables are immutable in Genshi. +.. _whitespace: + +--------------------------- +White-space and Line Breaks +--------------------------- + +Note that space or line breaks around directives is never automatically removed. +Consider the following example: + +.. code-block:: genshitext + + {% for item in items %} + {% if item.visible %} + ${item} + {% end %} + {% end %} + +This will result in two empty lines above and beneath every item, plus the +spaces used for indentation. If you want to supress a line break, simply end +the line with a backslash: + +.. code-block:: genshitext + + {% for item in items %}\ + {% if item.visible %}\ + ${item} + {% end %}\ + {% end %}\ + +Now there would be no empty lines between the items in the output. But you still +get the spaces used for indentation, and because the line breaks are removed, +they actually continue and add up between lines. There are numerous ways to +control white-space in the output while keeping the template readable, such as +moving the indentation into the delimiters, or moving the end delimiter on the +next line, and so on. + + .. _comments: -------- Comments -------- -Lines where the first non-whitespace characters are ``##`` are removed from -the output, and can thus be used for comments. This can be escaped using a +Parts in templates can be commented out using the delimiters ``{# ... #}``. +Any content in comments are removed from the output. + +.. code-block:: genshitext + + {# This won't end up in the output #} + This will. + +Just like directive delimiters, these can be escaped by prefixing with a backslash. + +.. code-block:: genshitext + + \{# This *will* end up in the output, including delimiters #} + This too. + + +.. _legacy: + +--------------------------- +Legacy Text Template Syntax +--------------------------- + +The syntax for text templates was redesigned in version 0.5 of Genshi to make +the language more flexible and powerful. The older syntax is based on line +starting with dollar signs, similar to e.g. Cheetah_ or Velocity_. + +.. _cheetah: http://cheetahtemplate.org/ +.. _velocity: http://jakarta.apache.org/velocity/ + +A simple template using the old syntax looked like this: + +.. code-block:: genshitext + + Dear $name, + + We have the following items for you: + #for item in items + * $item + #end + + All the best, + Foobar + +Beyond the requirement of putting directives on separate lines prefixed with +dollar signs, the language itself is very similar to the new one. Except that +comments are lines that start with two ``#`` characters, and a line-break at the +end of a directive is removed automatically. + +.. note:: If you're using this old syntax, it is strongly recommended to + migrate to the new syntax. Simply replace any references to + ``TextTemplate`` by ``NewTextTemplate`` (and also change the + text templates, of course). On the other hand, if you want to stick + with the old syntax for a while longer, replace references to + ``TextTemplate`` by ``OldTextTemplate``; while ``TextTemplate`` is + still an alias for the old language at this point, that will change + in a future release. But also note that the old syntax may be + dropped entirely in a future release.
new file mode 100644 --- /dev/null +++ b/doc/upgrade.txt @@ -0,0 +1,144 @@ +================ +Upgrading Genshi +================ + + +.. contents:: Contents + :depth: 2 +.. sectnum:: + + +------------------------------------ +Upgrading from Genshi 0.5.x to 0.6.x +------------------------------------ + +Required Python Version +----------------------- + +Support for Python 2.3 has been dropped in this release. Python 2.4 is +now the minimum version of Python required to run Genshi. + + +------------------------------------ +Upgrading from Genshi 0.4.x to 0.5.x +------------------------------------ + +Error Handling +-------------- + +The default error handling mode has been changed to "strict". This +means that accessing variables not defined in the template data will +now generate an immediate exception, as will accessing object +attributes or dictionary keys that don't exist. If your templates rely +on the old lenient behavior, you can configure Genshi to use that +instead. See the documentation for details on how to do that. But be +warned that lenient error handling may be removed completely in a +future release. + +Match Template Processing +------------------------- + +There has also been a subtle change to how ``py:match`` templates are +processed: in previous versions, all match templates would be applied +to the content generated by the matching template, and only the +matching template itself was applied recursively to the original +content. This behavior resulted in problems with many kinds of +recursive matching, and hence was changed for 0.5: now, all match +templates declared before the matching template are applied to the +original content, and match templates declared after the matching +template are applied to the generated content. This change should not +have any effect on most applications, but you may want to check your +use of match templates to make sure. + +Text Templates +-------------- + +Genshi 0.5 introduces a new, alternative syntax for text templates, +which is more flexible and powerful compared to the old syntax. For +backwards compatibility, this new syntax is not used by default, +though it will be in a future version. It is recommended that you +migrate to using this new syntax. To do so, simply rename any +references in your code to ``TextTemplate`` to ``NewTextTemplate``. To +explicitly use the old syntax, use ``OldTextTemplate`` instead, so +that you can be sure you'll be using the same language when the +default in Genshi is changed (at least until the old implementation is +completely removed). + +``Markup`` Constructor +---------------------- + +The ``Markup`` class no longer has a specialized constructor. The old +(undocumented) constructor provided a shorthand for doing positional +substitutions. If you have code like this: + +.. code-block:: python + + Markup('<b>%s</b>', name) + +You must replace it by the more explicit: + +.. code-block:: python + + Markup('<b>%s</b>') % name + +``Template`` Constructor +------------------------ + +The constructor of the ``Template`` class and its subclasses has changed +slightly: instead of the optional ``basedir`` parameter, it now expects +an (also optional) ``filepath`` parameter, which specifies the absolute +path to the template. You probably aren't using those constructors +directly, anyway, but using the ``TemplateLoader`` API instead. + + +------------------------------------ +Upgrading from Genshi 0.3.x to 0.4.x +------------------------------------ + +The modules ``genshi.filters`` and ``genshi.template`` have been +refactored into packages containing multiple modules. While code using +the regular APIs should continue to work without problems, you should +make sure to remove any leftover traces of the files ``filters.py`` +and ``template.py`` in the ``genshi`` package on the installation +path (including the corresponding ``.pyc`` files). This is not +necessary when Genshi was installed as a Python egg. + +Results of evaluating template expressions are no longer implicitly +called if they are callable. If you have been using that feature, you +will need to add the parenthesis to actually call the function. + +Instances of ``genshi.core.Attrs`` are now immutable. Filters +manipulating the attributes in a stream may need to be updated. Also, +the ``Attrs`` class no longer automatically wraps all attribute names +in ``QName`` objects, so users of the ``Attrs`` class need to do this +themselves. See the documentation of the ``Attrs`` class for more +information. + + +--------------------- +Upgrading from Markup +--------------------- + +Prior to version 0.3, the name of the Genshi project was "Markup". The +name change means that you will have to adjust your import statements +and the namespace URI of XML templates, among other things: + +* The package name was changed from "markup" to "genshi". Please + adjust any import statements referring to the old package name. +* The namespace URI for directives in Genshi XML templates has changed + from ``http://markup.edgewall.org/`` to + ``http://genshi.edgewall.org/``. Please update the ``xmlns:py`` + declaration in your template files accordingly. + +Furthermore, due to the inclusion of a text-based template language, +the class:: + + markup.template.Template + +has been renamed to:: + + genshi.template.MarkupTemplate + +If you've been using the Template class directly, you'll need to +update your code (a simple find/replace should do—the API itself +did not change).
--- a/doc/xml-templates.txt +++ b/doc/xml-templates.txt @@ -42,7 +42,9 @@ conditionals and looping, among others. To use directives in a template, the namespace must be declared, which is -usually done on the root element:: +usually done on the root element: + +.. code-block:: genshi <html xmlns="http://www.w3.org/1999/xhtml" xmlns:py="http://genshi.edgewall.org/" @@ -55,7 +57,9 @@ All directives can be applied as attributes, and some can also be used as elements. The ``if`` directives for conditionals, for example, can be used in -both ways:: +both ways: + +.. code-block:: genshi <html xmlns="http://www.w3.org/1999/xhtml" xmlns:py="http://genshi.edgewall.org/" @@ -67,7 +71,9 @@ ... </html> -This is basically equivalent to the following:: +This is basically equivalent to the following: + +.. code-block:: genshi <html xmlns="http://www.w3.org/1999/xhtml" xmlns:py="http://genshi.edgewall.org/" @@ -95,20 +101,34 @@ ``py:if`` --------- -The element is only rendered if the expression evaluates to a truth value:: +The element and its content is only rendered if the expression evaluates to a +truth value: + +.. code-block:: genshi <div> <b py:if="foo">${bar}</b> </div> Given the data ``foo=True`` and ``bar='Hello'`` in the template context, this -would produce:: +would produce: + +.. code-block:: xml <div> <b>Hello</b> </div> -This directive can also be used as an element:: +But setting ``foo=False`` would result in the following output: + +.. code-block:: xml + + <div> + </div> + +This directive can also be used as an element: + +.. code-block:: genshi <div> <py:if test="foo"> @@ -129,7 +149,9 @@ if no ``py:when`` branch matches, the ``py:otherwise`` branch is rendered. If the ``py:choose`` directive is empty the nested ``py:when`` directives will -be tested for truth:: +be tested for truth: + +.. code-block:: genshi <div py:choose=""> <span py:when="0 == 1">0</span> @@ -137,14 +159,18 @@ <span py:otherwise="">2</span> </div> -This would produce the following output:: +This would produce the following output: + +.. code-block:: xml <div> <span>1</span> </div> If the ``py:choose`` directive contains an expression the nested ``py:when`` -directives will be tested for equality to the parent ``py:choose`` value:: +directives will be tested for equality to the parent ``py:choose`` value: + +.. code-block:: genshi <div py:choose="1"> <span py:when="0">0</span> @@ -152,12 +178,23 @@ <span py:otherwise="">2</span> </div> -This would produce the following output:: +This would produce the following output: + +.. code-block:: xml <div> <span>1</span> </div> +These directives can also be used as elements: + +.. code-block:: genshi + + <py:choose test="1"> + <py:when test="0">0</py:when> + <py:when test="1">1</py:when> + <py:otherwise>2</py:otherwise> + </py:choose> Looping ======= @@ -167,19 +204,25 @@ ``py:for`` ---------- -The element is repeated for every item in an iterable:: +The element is repeated for every item in an iterable: + +.. code-block:: genshi <ul> <li py:for="item in items">${item}</li> </ul> -Given ``items=[1, 2, 3]`` in the context data, this would produce:: +Given ``items=[1, 2, 3]`` in the context data, this would produce: + +.. code-block:: xml <ul> <li>1</li><li>2</li><li>3</li> </ul> -This directive can also be used as an element:: +This directive can also be used as an element: + +.. code-block:: genshi <ul> <py:for each="item in items"> @@ -199,7 +242,9 @@ The ``py:def`` directive can be used to create macros, i.e. snippets of template code that have a name and optionally some parameters, and that can be -inserted in other places:: +inserted in other places: + +.. code-block:: genshi <div> <p py:def="greeting(name)" class="greeting"> @@ -209,7 +254,9 @@ ${greeting('everyone else')} </div> -The above would be rendered to:: +The above would be rendered to: + +.. code-block:: xml <div> <p class="greeting"> @@ -221,7 +268,9 @@ </div> If a macro doesn't require parameters, it can be defined without the -parenthesis. For example:: +parenthesis. For example: + +.. code-block:: genshi <div> <p py:def="greeting" class="greeting"> @@ -230,7 +279,9 @@ ${greeting()} </div> -The above would be rendered to:: +The above would be rendered to: + +.. code-block:: xml <div> <p class="greeting"> @@ -238,7 +289,9 @@ </p> </div> -This directive can also be used as an element:: +This directive can also be used as an element: + +.. code-block:: genshi <div> <py:def function="greeting(name)"> @@ -258,7 +311,9 @@ content. For example, the match template defined in the following template matches any -element with the tag name “greeting”:: +element with the tag name “greeting”: + +.. code-block:: genshi <div> <span py:match="greeting"> @@ -267,7 +322,9 @@ <greeting name="Dude" /> </div> -This would result in the following output:: +This would result in the following output: + +.. code-block:: xml <div> <span> @@ -282,7 +339,15 @@ .. _`Using XPath`: streams.html#using-xpath -This directive can also be used as an element:: +Match templates are applied both to the original markup as well to the +generated markup. The order in which they are applied depends on the order +they are declared in the template source: a match template defined after +another match template is applied to the output generated by the first match +template. The match templates basically form a pipeline. + +This directive can also be used as an element: + +.. code-block:: genshi <div> <py:match path="greeting"> @@ -291,6 +356,54 @@ <greeting name="Dude" /> </div> +When used this way, the ``py:match`` directive can also be annotated with a +couple of optimization hints. For example, the following informs the matching +engine that the match should only be applied once: + +.. code-block:: genshi + + <py:match path="body" once="true"> + <body py:attrs="select('@*')"> + <div id="header">...</div> + ${select("*|text()")} + <div id="footer">...</div> + </body> + </py:match> + +The following optimization hints are recognized: + ++---------------+-----------+-----------------------------------------------+ +| Attribute | Default | Description | ++===============+===========+===============================================+ +| ``buffer`` | ``true`` | Whether the matched content should be | +| | | buffered in memory. Buffering can improve | +| | | performance a bit at the cost of needing more | +| | | memory during rendering. Buffering is | +| | | ''required'' for match templates that contain | +| | | more than one invocation of the ``select()`` | +| | | function. If there is only one call, and the | +| | | matched content can potentially be very long, | +| | | consider disabling buffering to avoid | +| | | excessive memory use. | ++---------------+-----------+-----------------------------------------------+ +| ``once`` | ``false`` | Whether the engine should stop looking for | +| | | more matching elements after the first match. | +| | | Use this on match templates that match | +| | | elements that can only occur once in the | +| | | stream, such as the ``<head>`` or ``<body>`` | +| | | elements in an HTML template, or elements | +| | | with a specific ID. | ++---------------+-----------+-----------------------------------------------+ +| ``recursive`` | ``true`` | Whether the match template should be applied | +| | | to its own output. Note that ``once`` implies | +| | | non-recursive behavior, so this attribute | +| | | only needs to be set for match templates that | +| | | don't also have ``once`` set. | ++---------------+-----------+-----------------------------------------------+ + +.. note:: The ``py:match`` optimization hints were added in the 0.5 release. In + earlier versions, the attributes have no effect. + Variable Binding ================ @@ -306,19 +419,25 @@ than once, and that actually results in a database query, assigning the results to a variable using this directive would probably help. -For example:: +For example: + +.. code-block:: genshi <div> <span py:with="y=7; z=x+10">$x $y $z</span> </div> -Given ``x=42`` in the context data, this would produce:: +Given ``x=42`` in the context data, this would produce: + +.. code-block:: xml <div> <span>42 7 52</span> </div> -This directive can also be used as an element:: +This directive can also be used as an element: + +.. code-block:: genshi <div> <py:with vars="y=7; z=x+10">$x $y $z</py:with> @@ -338,21 +457,27 @@ ``py:attrs`` ------------ -This directive adds, modifies or removes attributes from the element:: +This directive adds, modifies or removes attributes from the element: + +.. code-block:: genshi <ul> <li py:attrs="foo">Bar</li> </ul> Given ``foo={'class': 'collapse'}`` in the template context, this would -produce:: +produce: + +.. code-block:: xml <ul> <li class="collapse">Bar</li> </ul> Attributes with the value ``None`` are omitted, so given ``foo={'class': None}`` -in the context for the same template this would produce:: +in the context for the same template this would produce: + +.. code-block:: xml <ul> <li>Bar</li> @@ -367,13 +492,17 @@ -------------- This directive replaces any nested content with the result of evaluating the -expression:: +expression: + +.. code-block:: genshi <ul> <li py:content="bar">Hello</li> </ul> -Given ``bar='Bye'`` in the context data, this would produce:: +Given ``bar='Bye'`` in the context data, this would produce: + +.. code-block:: xml <ul> <li>Bye</li> @@ -388,19 +517,30 @@ -------------- This directive replaces the element itself with the result of evaluating the -expression:: +expression: + +.. code-block:: genshi <div> <span py:replace="bar">Hello</span> </div> -Given ``bar='Bye'`` in the context data, this would produce:: +Given ``bar='Bye'`` in the context data, this would produce: + +.. code-block:: xml <div> Bye </div> -This directive can only be used as an attribute. +This directive can also be used as an element (since version 0.5): + +.. code-block:: genshi + + <div> + <py:replace value="title">Placeholder</py:replace> + </div> + .. _`py:strip`: @@ -410,13 +550,17 @@ This directive conditionally strips the top-level element from the output. When the value of the ``py:strip`` attribute evaluates to ``True``, the element is -stripped from the output:: +stripped from the output: + +.. code-block:: genshi <div> <div py:strip="True"><b>foo</b></div> </div> -This would be rendered as:: +This would be rendered as: + +.. code-block:: xml <div> <b>foo</b> @@ -462,7 +606,9 @@ For this, you need to declare the XInclude namespace (commonly bound to the prefix “xi”) and use the ``<xi:include>`` element where you want the external -file to be pulled in:: +file to be pulled in: + +.. code-block:: genshi <html xmlns="http://www.w3.org/1999/xhtml" xmlns:py="http://genshi.edgewall.org/" @@ -485,7 +631,9 @@ By default, an error will be raised if an included file is not found. If that's not what you want, you can specify fallback content that should be used if the include fails. For example, to to make the include above fail silently, you'd -write:: +write: + +.. code-block:: genshi <xi:include href="base.html"><xi:fallback /></xi:include> @@ -494,32 +642,61 @@ .. _`xinclude specification`: http://www.w3.org/TR/xinclude/ + +Dynamic Includes +================ + Incudes in Genshi are fully dynamic: Just like normal attributes, the `href` attribute accepts expressions, and directives_ can be used on the ``<xi:include />`` element just as on any other element, meaning you can do -things like conditional includes:: +things like conditional includes: + +.. code-block:: genshi <xi:include href="${name}.html" py:if="not in_popup" py:for="name in ('foo', 'bar', 'baz')" /> +Including Text Templates +======================== + +The ``parse`` attribute of the ``<xi:include>`` element can be used to specify +whether the included template is an XML template or a text template (using the +new syntax added in Genshi 0.5): + +.. code-block:: genshi + + <xi:include href="myscript.js" parse="text" /> + +This example would load the ``myscript.js`` file as a ``NewTextTemplate``. See +`text templates`_ for details on the syntax of text templates. + +.. _`text templates`: text-templates.html + + .. _comments: -------- Comments -------- -Normal XML/HTML comment syntax can be used in templates:: +Normal XML/HTML comment syntax can be used in templates: + +.. code-block:: genshi <!-- this is a comment --> However, such comments get passed through the processing pipeline and are by default included in the final output. If that's not desired, prefix the comment -text with an exclamation mark:: +text with an exclamation mark: + +.. code-block:: genshi <!-- !this is a comment too, but one that will be stripped from the output --> Note that it does not matter whether there's whitespace before or after the -exclamation mark, so the above could also be written as follows:: +exclamation mark, so the above could also be written as follows: + +.. code-block:: genshi <!--! this is a comment too, but one that will be stripped from the output -->
--- a/doc/xpath.txt +++ b/doc/xpath.txt @@ -52,8 +52,8 @@ * ``sum()`` The mathematical operators (``+``, ``-``, ``*``, ``div``, and ``mod``) are not -yet supported, whereas the various comparison and logical operators should work -as expected. +yet supported, whereas sub-expressions and the various comparison and logical +operators should work as expected. You can also use XPath variable references (``$var``) inside predicates. @@ -62,25 +62,34 @@ Querying Streams ---------------- -:: - - from genshi.input import XML +The ``Stream`` class provides a ``select(path)`` function that can be used to +retrieve subsets of the stream: - doc = XML('''<doc> - <items count="2"> - <item status="new"> - <summary>Foo</summary> - </item> - <item status="closed"> - <summary>Bar</summary> - </item> - </items> - </doc>''') - print doc.select('items/item[@status="closed"]/summary/text()') +.. code-block:: pycon -This would result in the following output:: + >>> from genshi.input import XML - Bar + >>> doc = XML('''<doc> + ... <items count="4"> + ... <item status="new"> + ... <summary>Foo</summary> + ... </item> + ... <item status="closed"> + ... <summary>Bar</summary> + ... </item> + ... <item status="closed" resolution="invalid"> + ... <summary>Baz</summary> + ... </item> + ... <item status="closed" resolution="fixed"> + ... <summary>Waz</summary> + ... </item> + ... </items> + ... </doc>''') + + >>> print doc.select('items/item[@status="closed" and ' + ... '(@resolution="invalid" or not(@resolution))]/summary/text()') + BarBaz + ---------------------
--- a/examples/bench/basic.py +++ b/examples/bench/basic.py @@ -9,7 +9,8 @@ import sys import timeit -__all__ = ['clearsilver', 'myghty', 'django', 'kid', 'genshi', 'cheetah'] +__all__ = ['clearsilver', 'mako', 'django', 'kid', 'genshi', 'genshi_text', + 'simpletal'] def genshi(dirname, verbose=False): from genshi.template import TemplateLoader @@ -24,19 +25,28 @@ print render() return render -def myghty(dirname, verbose=False): - try: - from myghty import interp - except ImportError: - print>>sys.stderr, 'Mighty not installed, skipping' - return lambda: None - interpreter = interp.Interpreter(component_root=dirname) +def genshi_text(dirname, verbose=False): + from genshi.core import escape + from genshi.template import TemplateLoader, NewTextTemplate + loader = TemplateLoader([dirname], auto_reload=False) + template = loader.load('template.txt', cls=NewTextTemplate) + def render(): + data = dict(escape=escape, title='Just a test', user='joe', + items=['Number %d' % num for num in range(1, 15)]) + return template.generate(**data).render('text') + + if verbose: + print render() + return render + +def mako(dirname, verbose=False): + from mako.lookup import TemplateLookup + lookup = TemplateLookup(directories=[dirname], filesystem_checks=False) + template = lookup.get_template('template.html') def render(): data = dict(title='Just a test', user='joe', - items=['Number %d' % num for num in range(1, 15)]) - buffer = StringIO() - interpreter.execute("template.myt", request_args=data, out_buffer=buffer) - return buffer.getvalue() + list_items=['Number %d' % num for num in range(1, 15)]) + return template.render(**data) if verbose: print render() return render @@ -112,7 +122,7 @@ try: import kid except ImportError: - print>>sys.stderr, "SimpleTAL not installed, skipping" + print>>sys.stderr, "Kid not installed, skipping" return lambda: None kid.path = kid.TemplatePath([dirname]) template = kid.load_template('template.kid').Template @@ -180,12 +190,15 @@ verbose = '-v' in sys.argv if '-p' in sys.argv: - import hotshot, hotshot.stats - prof = hotshot.Profile("template.prof") - benchtime = prof.runcall(run, engines, number=100, verbose=verbose) - stats = hotshot.stats.load("template.prof") + import cProfile, pstats + prof = cProfile.Profile() + prof.run('run(%r, number=200, verbose=%r)' % (engines, verbose)) + stats = pstats.Stats(prof) stats.strip_dirs() - stats.sort_stats('time', 'calls') - stats.print_stats() + stats.sort_stats('calls') + stats.print_stats(25) + if verbose: + stats.print_callees() + stats.print_callers() else: run(engines, verbose=verbose)
--- a/examples/bench/bigtable.py +++ b/examples/bench/bigtable.py @@ -10,7 +10,7 @@ import timeit from StringIO import StringIO from genshi.builder import tag -from genshi.template import MarkupTemplate +from genshi.template import MarkupTemplate, NewTextTemplate try: from elementtree import ElementTree as et @@ -41,9 +41,9 @@ DjangoContext = DjangoTemplate = None try: - from myghty.interp import Interpreter as MyghtyInterpreter + from mako.template import Template as MakoTemplate except ImportError: - MyghtyInterpreter = None + MakoTemplate = None table = [dict(a=1,b=2,c=3,d=4,e=5,f=6,g=7,h=8,i=9,j=10) for x in range(1000)] @@ -60,6 +60,14 @@ <table xmlns:py="http://genshi.edgewall.org/">$table</table> """) +genshi_text_tmpl = NewTextTemplate(""" +<table> +{% for row in table %}<tr> +{% for c in row.values() %}<td>$c</td>{% end %} +</tr>{% end %} +</table> +""") + if DjangoTemplate: django_tmpl = DjangoTemplate(""" <table> @@ -74,29 +82,32 @@ context = DjangoContext({'table': table}) django_tmpl.render(context) -if MyghtyInterpreter: - interpreter = MyghtyInterpreter() - component = interpreter.make_component(""" +if MakoTemplate: + mako_tmpl = MakoTemplate(""" <table> -% for row in ARGS['table']: - <tr> -% for col in row.values(): - <td><% col %></td> -% -% - </tr> + % for row in table: + <tr> + % for col in row.values(): + <td>${ col | h }</td> + % endfor + </tr> + % endfor </table> """) - def test_myghty(): - """Myghty Template""" - buf = StringIO() - interpreter.execute(component, request_args={'table':table}, out_buffer=buf) + def test_mako(): + """Mako Template""" + mako_tmpl.render(table=table) def test_genshi(): """Genshi template""" stream = genshi_tmpl.generate(table=table) stream.render('html', strip_whitespace=False) +def test_genshi_text(): + """Genshi text template""" + stream = genshi_text_tmpl.generate(table=table) + stream.render('text') + def test_genshi_builder(): """Genshi template + tag builder""" stream = tag.TABLE([ @@ -185,9 +196,9 @@ def run(which=None, number=10): - tests = ['test_builder', 'test_genshi', 'test_genshi_builder', - 'test_myghty', 'test_kid', 'test_kid_et', 'test_et', 'test_cet', - 'test_clearsilver', 'test_django'] + tests = ['test_builder', 'test_genshi', 'test_genshi_text', + 'test_genshi_builder', 'test_mako', 'test_kid', 'test_kid_et', + 'test_et', 'test_cet', 'test_clearsilver', 'test_django'] if which: tests = filter(lambda n: n[5:] in which, tests) @@ -208,12 +219,15 @@ which = [arg for arg in sys.argv[1:] if arg[0] != '-'] if '-p' in sys.argv: - import hotshot, hotshot.stats - prof = hotshot.Profile("template.prof") - benchtime = prof.runcall(run, which, number=1) - stats = hotshot.stats.load("template.prof") + import cProfile, pstats + prof = cProfile.Profile() + prof.run('run(%r, number=1)' % which) + stats = pstats.Stats(prof) stats.strip_dirs() stats.sort_stats('time', 'calls') - stats.print_stats() + stats.print_stats(25) + if '-v' in sys.argv: + stats.print_callees() + stats.print_callers() else: run(which)
--- a/examples/bench/genshi/base.html +++ b/examples/bench/genshi/base.html @@ -6,12 +6,12 @@ Hello, ${name}! </p> - <body py:match="body"> + <py:match path="body" once="true"><body> <div id="header"> <h1>${title}</h1> </div> ${select('*')} <div id="footer" /> - </body> + </body></py:match> </html>
new file mode 100644 --- /dev/null +++ b/examples/bench/genshi_text/footer.txt @@ -0,0 +1,1 @@ +<div id="footer"></div>
new file mode 100644 --- /dev/null +++ b/examples/bench/genshi_text/header.txt @@ -0,0 +1,3 @@ +<div id="header"> + <h1>${escape(title)}</h1> +</div>
new file mode 100644 --- /dev/null +++ b/examples/bench/genshi_text/template.txt @@ -0,0 +1,28 @@ +<!DOCTYPE html + PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" + "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml" lang="en"> + +<head> + <title>${escape(title)}</title> +</head> + +<body> + {% include header.txt %} + {% def greeting(name) %} + Hello, ${name}! + {% end %} + + <div>${greeting(user)}</div> + <div>${greeting("me")}</div> + <div>${greeting("world")}</div> + + <h2>Loop</h2> + {% if items %}<ul> + {% for idx, item in enumerate(items) %}\ + <li{% if idx + 1 == len(items) %} class="last"{% end %}>${escape(item)}</li> + {% end %} + </ul>{% end %} + + {% include footer.txt %} +</body>
new file mode 100644 --- /dev/null +++ b/examples/bench/mako/footer.html @@ -0,0 +1,2 @@ +<div id="footer"> +</div>
new file mode 100644 --- /dev/null +++ b/examples/bench/mako/header.html @@ -0,0 +1,5 @@ +<div id="header"> + <h1>${title | h}</h1> +</div> + +
new file mode 100644 --- /dev/null +++ b/examples/bench/mako/template.html @@ -0,0 +1,29 @@ +<!DOCTYPE html + PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" + "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml" lang="en"> + <head> + <title>${title}</title> + </head> + <body> + <%def name="greeting(name)"> + <p>Hello, ${name | h}!</p> + </%def> + + <%include file="header.html"/> + + ${greeting(user)} + ${greeting('me')} + ${greeting('world')} + + <h2>Loop</h2> + % if items: + <ul> + % for idx, item in enumerate(items): + <li ${idx+1==len(items) and "class='last'" or ""}>${item | h}</li> + % endfor + </ul> + % endif + <%include file="footer.html"/> + </body> +</html>
new file mode 100644 --- /dev/null +++ b/examples/bench/xpath.py @@ -0,0 +1,170 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2006-2008 Edgewall Software +# All rights reserved. +# +# This software is licensed as described in the file COPYING, which +# you should have received as part of this distribution. The terms +# are also available at http://genshi.edgewall.org/wiki/License. +# +# This software consists of voluntary contributions made by many +# individuals. For the exact contribution history, see the revision +# history and logs, available at http://genshi.edgewall.org/log/. + + +try: + from os import times + def time_func(): + tup = times() + #just user time + return tup[0] # + tup[1] +except ImportError: + from time import time as time_func + +from genshi.core import START, END +from genshi.path import Path +from genshi.input import XML + +def benchmark(f, acurate_time=1): + """Checks how much time does function f work. It runs it as + many times as needed for avoiding inaccuracy""" + + runs = 1 + while True: + start_time = time_func() + for _ in xrange(runs): + f() + dt = time_func() - start_time + if dt >= acurate_time: + break + runs *= 2 + return dt / runs + +def spell(t): + """Returns spelled representation of time""" + units = [(0.000001, 'microsecond', 'microseconds'), + (0.001, 'milisecond', 'miliseconds'), + (1, 'second', 'seconds'), + (60, 'minute', 'minutes'), + (60*60, 'hour', 'hours'), + ] + i = 0 + at = abs(t) + while i + 1 < len(units) and at >= units[i + 1][0]: + i += 1 + t /= units[i][0] + if t >= 2: + name = units[i][2] + else: + name = units[i][1] + return "%f %s"%(t, name) + +def test_paths_in_streams(exprs, streams, test_strategies=False): + for expr in exprs: + print "Testing path %r" % expr + for stream, sname in streams: + print '\tRunning on "%s" example:' % sname + + path = Path(expr) + def f(): + for e in path.select(stream): + pass + t = spell(benchmark(f)) + print "\t\tselect:\t\t%s" % t + + def f(): + path = Path(expr) + for e in path.select(stream): + pass + t = spell(benchmark(f)) + print "\t\tinit + select:\t%s" % t + + if test_strategies and len(path.paths) == 1: + from genshi.path import GenericStrategy, SingleStepStrategy, \ + SimplePathStrategy + from genshi.tests.path import FakePath + strategies = (GenericStrategy, SingleStepStrategy, + SimplePathStrategy) + for strategy in strategies: + if not strategy.supports(path.paths[0]): + continue + print "\t\t%s Strategy"%strategy.__name__ + fp = FakePath(strategy(path.paths[0])) + def f(): + for e in fp.select(stream): + pass + t = spell(benchmark(f)) + print "\t\t\tselect:\t\t%s"%t + + +def test_documents(test_strategies=False): + streams = [] + + s = XML("""\ +<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="pl" xmlns:py="http://genshi.edgewall.org/" py:strip="" lang="en"> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> + <title>Foo</title> + </head> + <body> + <h1>Hello</h1> + </body> +</html> +""") + streams.append((s, "small document")) + + s = XML("""\ +<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="pl" xmlns:py="http://genshi.edgewall.org/" py:strip="" lang="en"> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> + <title>Foo</title> + </head> + <body> + <h1>Hello</h1> + <div id="splash"> + <ul> + <li><a class="b1" href="http://genshi.edgewall.org/"> + <strong>Genshi</strong> + Python toolkit for generating output for the web</a></li> + <li><a class="b2" href="http://babel.edgewall.org/"> + <strong>Babel</strong> + Python library for I18n/L10n in web applications</a></li> + <li><a class="b3" href="http://bitten.edgewall.org/"> + <strong>Bitten</strong> + Continuous integration plugin for Trac</a></li> + <li><a class="b4" href="http://posterity.edgewall.org/"> + <strong>Posterity</strong> + Web-based email system</a></li> + </ul> + <div id="trac-splash"> + <a href="http://trac.edgewall.org/"> + <strong>Trac</strong> Web-based lightweight project management + system + </a> + </div> + </div> + </body> +</html> +""") + streams.append((s, "big document")) + + paths = [ + '.', + '*|text()', + 'html', + 'html[@lang="en"]', + 'html/body/h1/text()', + 'html/body/div/a/@href', + 'html/body/div[@id="splash"]/a[@class="b4"]/strong/text()', + 'descendant-or-self::text()', + 'descendant-or-self::h1/text()', + ] + test_paths_in_streams(paths, streams, test_strategies) + +if __name__ == '__main__': + from sys import argv + if "--strategies" in argv: + test_strategies = True + else: + test_strategies = False + test_documents(test_strategies)
deleted file mode 100644 --- a/examples/cherrypy/config.txt +++ /dev/null @@ -1,3 +0,0 @@ -[global] -server.socket_port = 8000 -server.thread_pool = 10
deleted file mode 100644 --- a/examples/cherrypy/index.html +++ /dev/null @@ -1,11 +0,0 @@ -<!DOCTYPE html - PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" - "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" - xmlns:py="http://genshi.edgewall.org/" - xmlns:xi="http://www.w3.org/2001/XInclude" - lang="en"> - <body> - <span class="greeting">Hello, ${name}!</span> - </body> -</html>
deleted file mode 100644 --- a/examples/cherrypy/index.py +++ /dev/null @@ -1,19 +0,0 @@ -import os -import sys - -import cherrypy -from genshi.template import TemplateLoader - -loader = TemplateLoader([os.path.dirname(os.path.abspath(__file__))]) - - -class Example(object): - - @cherrypy.expose - def index(self): - tmpl = loader.load('index.html') - return tmpl.generate(name='world').render('xhtml') - - -if __name__ == '__main__': - cherrypy.quickstart(Example(), config='config.txt')
deleted file mode 100644 --- a/examples/includes/common/macros.html +++ /dev/null @@ -1,12 +0,0 @@ -<div xmlns:py="http://genshi.edgewall.org/" - py:strip=""> - <div py:def="macro1">reference me, please</div> - <div py:def="macro2(name, classname='expanded')" class="${classname}"> - Hello ${name.title()} - </div> - <span py:match="greeting" class="greeting"> - Hello ${select('@name')} - </span> - <span py:match="span[@class='greeting']" style="text-decoration: underline" - py:content="select('text()')"/> -</div>
deleted file mode 100644 --- a/examples/includes/module/test.html +++ /dev/null @@ -1,22 +0,0 @@ -<!DOCTYPE html - PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" - "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" - xmlns:py="http://genshi.edgewall.org/" - xmlns:xi="http://www.w3.org/2001/XInclude"> - <xi:include href="${skin}/layout.html" /> - <xi:include href="custom_stuff.html"><xi:fallback/></xi:include> - <body class="$bozz"> - <ul py:attrs="{'id': 'second', 'class': None}" py:if="len(items) > 0"> - <li py:for="item in items">Item ${item.split()[-1]}</li> - XYZ ${hey} - </ul> - ${macro1()} ${macro1()} ${macro1()} - ${macro2('john')} - ${macro2('kate', classname='collapsed')} - <div py:content="macro2('helmut')" py:strip="">Replace me</div> - <greeting name="Dude" /> - <greeting name="King" /> - <span class="greeting">Hello Silicon</span> - </body> -</html>
deleted file mode 100755 --- a/examples/includes/run.py +++ /dev/null @@ -1,49 +0,0 @@ -#!/usr/bin/python -# -*- coding: utf-8 -*- - -import os -import sys -import timing - -from genshi.template import Context, TemplateLoader - -def test(): - base_path = os.path.dirname(os.path.abspath(__file__)) - loader = TemplateLoader([os.path.join(base_path, 'skins'), - os.path.join(base_path, 'module'), - os.path.join(base_path, 'common')]) - - timing.start() - tmpl = loader.load('test.html') - timing.finish() - print ' --> parse stage: %dms' % timing.milli() - - data = dict(hello='<world>', skin='default', hey='ZYX', bozz=None, - items=['Number %d' % num for num in range(1, 15)]) - - print tmpl.generate(Context(**data)).render(method='html') - - times = [] - for i in range(100): - timing.start() - list(tmpl.generate(Context(**data))) - timing.finish() - times.append(timing.milli()) - sys.stdout.write('.') - sys.stdout.flush() - print - - print ' --> render stage: %dms (avg), %dms (min), %dms (max)' % ( - sum(times) / len(times), min(times), max(times)) - -if __name__ == '__main__': - if '-p' in sys.argv: - import hotshot, hotshot.stats - prof = hotshot.Profile("template.prof") - benchtime = prof.runcall(test) - stats = hotshot.stats.load("template.prof") - stats.strip_dirs() - stats.sort_stats('time', 'calls') - stats.print_stats() - else: - test()
deleted file mode 100644 --- a/examples/includes/skins/default/footer.html +++ /dev/null @@ -1,4 +0,0 @@ -<div id="footer"> - <hr /> - <h1>And goodbye</h1> -</div>
deleted file mode 100644 --- a/examples/includes/skins/default/header.html +++ /dev/null @@ -1,3 +0,0 @@ -<div id="masthead"> - <h1>Welcome</h1> -</div> \ No newline at end of file
deleted file mode 100644 --- a/examples/includes/skins/default/layout.html +++ /dev/null @@ -1,17 +0,0 @@ -<html xmlns="http://www.w3.org/1999/xhtml" - xmlns:py="http://genshi.edgewall.org/" - xmlns:xi="http://www.w3.org/2001/XInclude" - py:strip=""> - <xi:include href="../macros.html" /> - <head> - <title>Hello ${hello}</title> - <style type="text/css">@import(style.css)</style> - </head> - <body py:match="body"> - <xi:include href="header.html" /> - <div id="content"> - ${select('*')} - </div> - <xi:include href="footer.html" /> - </body> -</html>
deleted file mode 100644 --- a/examples/transform/README.txt +++ /dev/null @@ -1,3 +0,0 @@ -This example shows how to transform some HTML input, while adding -elements such as headers, and using <em>/<strong> instead of <i>/<b>. -The output is that a proper XTHML document.
deleted file mode 100644 --- a/examples/transform/index.html +++ /dev/null @@ -1,24 +0,0 @@ -<HTML> - <HEAD> - <TITLE>Aaarrgh</TITLE> - <LINK REL=stylesheet href='badstyle.css'> - </HEAD> - - <BODY> - <H1>Aaargh</H1> - <P> - <B>Lorem <I>ipsum</I></B> dolor sit amet, consectetur<BR> - adipisicing elit, sed do eiusmod tempor incididunt ut<BR> - labore et dolore magna aliqua. Ut enim ad minim veniam,<BR> - quis nostrud exercitation ullamco laboris nisi ut<BR> - aliquip ex ea commodo consequat. - </P> - <P> - Duis aute irure dolor in reprehenderit in voluptate velit<BR> - esse cillum dolore eu fugiat nulla pariatur. Excepteur sint<BR> - occaecat cupidatat non proident, sunt in culpa qui officia<BR> - deserunt mollit anim <I>id est laborum</I>. - </P> - </BODY> - -</HTML>
deleted file mode 100644 --- a/examples/transform/run.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/python -# -*- coding: utf-8 -*- - -import os -import sys - -from genshi.input import HTMLParser -from genshi.template import Context, MarkupTemplate - -def transform(html_filename, tmpl_filename): - tmpl_fileobj = open(tmpl_filename) - tmpl = MarkupTemplate(tmpl_fileobj, tmpl_filename) - tmpl_fileobj.close() - - html_fileobj = open(html_filename) - html = HTMLParser(html_fileobj, html_filename) - print tmpl.generate(Context(input=html)).render('xhtml') - html_fileobj.close() - -if __name__ == '__main__': - basepath = os.path.dirname(os.path.abspath(__file__)) - tmpl_filename = os.path.join(basepath, 'template.xml') - html_filename = os.path.join(basepath, 'index.html') - transform(html_filename, tmpl_filename)
deleted file mode 100644 --- a/examples/transform/template.xml +++ /dev/null @@ -1,22 +0,0 @@ -<!DOCTYPE html - PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" - "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> -<html xmlns:py="http://genshi.edgewall.org/" py:strip=""> - - <!--! Add a header DIV on top of every page with a logo image --> - <body py:match="BODY|body"> - <div id="header"> - <img src="logo.png" alt="Bad Style"/> - </div> - ${select('*')} - </body> - - <!--! Use semantic instead of presentational tags for emphasis --> - <strong py:match="B|b">${select('*|text()')}</strong> - <em py:match="I|i">${select('*|text()')}</em> - - <!--! Include the actual HTML stream, which will be processed by the rules - defined above --> - ${input} - -</html>
deleted file mode 100644 --- a/examples/turbogears/README.txt +++ /dev/null @@ -1,4 +0,0 @@ -GenshiTest - -This is a TurboGears (http://www.turbogears.org) project. It can be -started by running the start-markuptest.py script.
deleted file mode 100644 --- a/examples/turbogears/dev.cfg +++ /dev/null @@ -1,63 +0,0 @@ -[global] -# This is where all of your settings go for your development environment -# Settings that are the same for both development and production -# (such as template engine, encodings, etc.) all go in -# genshitest/config/app.cfg - -# DATABASE - -# pick the form for your database -# sqlobject.dburi="postgres://username@hostname/databasename" -# sqlobject.dburi="mysql://username:password@hostname:port/databasename" -# sqlobject.dburi="sqlite:///file_name_and_path" - -# If you have sqlite, here's a simple default to get you started -# in development -sqlobject.dburi="sqlite://%(current_dir_uri)s/devdata.sqlite" - - -# if you are using a database or table type without transactions -# (MySQL default, for example), you should turn off transactions -# by prepending notrans_ on the uri -# sqlobject.dburi="notrans_mysql://username:password@hostname:port/databasename" - -# for Windows users, sqlite URIs look like: -# sqlobject.dburi="sqlite:///drive_letter:/path/to/file" - -# SERVER - -# Some server parameters that you may want to tweak -# server.socket_port=8080 - -# Enable the debug output at the end on pages. -# log_debug_info_filter.on = False - -server.environment="development" -autoreload.package="genshitest" - -# Set to True if you'd like to abort execution if a controller gets an -# unexpected parameter. False by default -tg.strict_parameters = True - -# LOGGING -# Logging configuration generally follows the style of the standard -# Python logging module configuration. Note that when specifying -# log format messages, you need to use *() for formatting variables. -# Deployment independent log configuration is in genshitest/config/log.cfg -[logging] - -[[loggers]] -[[[geshnitest]]] -level='DEBUG' -qualname='genshitest' -handlers=['debug_out'] - -[[[allinfo]]] -level='INFO' -handlers=['debug_out'] - -[[[access]]] -level='INFO' -qualname='turbogears.access' -handlers=['access_out'] -propagate=0
deleted file mode 100644 --- a/examples/turbogears/genshitest/config/app.cfg +++ /dev/null @@ -1,121 +0,0 @@ -[global] -# The settings in this file should not vary depending on the deployment -# environment. dev.cfg and prod.cfg are the locations for -# the different deployment settings. Settings in this file will -# be overridden by settings in those other files. - -# The commented out values below are the defaults - -# VIEW - -# which view (template engine) to use if one is not specified in the -# template name -tg.defaultview = "genshi" - -# The following kid settings determine the settings used by the kid serializer. - -# One of (html|xml|json) -# kid.outputformat="html" - -# kid.encoding="utf-8" - -# The sitetemplate is used for overall styling of a site that -# includes multiple TurboGears applications -# tg.sitetemplate="<packagename.templates.templatename>" - -# Allow every exposed function to be called as json, -# tg.allow_json = False - -# List of Widgets to include on every page. -# for exemple ['turbogears.mochikit'] -# tg.include_widgets = [] - -# Set to True if the scheduler should be started -# tg.scheduler = False - -# VISIT TRACKING -# Each visit to your application will be assigned a unique visit ID tracked via -# a cookie sent to the visitor's browser. -# -------------- - -# Enable Visit tracking -visit.on=True - -# Number of minutes a visit may be idle before it expires. -# visit.timeout=20 - -# The name of the cookie to transmit to the visitor's browser. -# visit.cookie.name="tg-visit" - -# Domain name to specify when setting the cookie (must begin with . according to -# RFC 2109). The default (None) should work for most cases and will default to -# the machine to which the request was made. NOTE: localhost is NEVER a valid -# value and will NOT WORK. -# visit.cookie.domain=None - -# Specific path for the cookie -# visit.cookie.path="/" - -# The name of the VisitManager plugin to use for visitor tracking. -visit.manager="sqlobject" - -# Database class to use for visit tracking -visit.soprovider.model = "genshitest.model.Visit" - -# IDENTITY -# General configuration of the TurboGears Identity management module -# -------- - -# Switch to turn on or off the Identity management module -identity.on=True - -# [REQUIRED] URL to which CherryPy will internally redirect when an access -# control check fails. If Identity management is turned on, a value for this -# option must be specified. -identity.failure_url="/login" - -# identity.provider='sqlobject' - -# The names of the fields on the login form containing the visitor's user ID -# and password. In addition, the submit button is specified simply so its -# existence may be stripped out prior to passing the form data to the target -# controller. -# identity.form.user_name="user_name" -# identity.form.password="password" -# identity.form.submit="login" - -# What sources should the identity provider consider when determining the -# identity associated with a request? Comma separated list of identity sources. -# Valid sources: form, visit, http_auth -# identity.source="form,http_auth,visit" - -# SqlObjectIdentityProvider -# Configuration options for the default IdentityProvider -# ------------------------- - -# The classes you wish to use for your Identity model. Remember to not use reserved -# SQL keywords for class names (at least unless you specify a different table -# name using sqlmeta). -identity.soprovider.model.user="genshitest.model.User" -identity.soprovider.model.group="genshitest.model.Group" -identity.soprovider.model.permission="genshitest.model.Permission" - -# The password encryption algorithm used when comparing passwords against what's -# stored in the database. Valid values are 'md5' or 'sha1'. If you do not -# specify an encryption algorithm, passwords are expected to be clear text. -# -# The SqlObjectProvider *will* encrypt passwords supplied as part of your login -# form. If you set the password through the password property, like: -# my_user.password = 'secret' -# the password will be encrypted in the database, provided identity is up and -# running, or you have loaded the configuration specifying what encryption to -# use (in situations where identity may not yet be running, like tests). - -# identity.soprovider.encryption_algorithm=None -[/static] -static_filter.on = True -static_filter.dir = "%(top_level_dir)s/static" - -[/favicon.ico] -static_filter.on = True -static_filter.file = "%(top_level_dir)s/static/images/favicon.ico"
deleted file mode 100644 --- a/examples/turbogears/genshitest/config/log.cfg +++ /dev/null @@ -1,29 +0,0 @@ -# LOGGING -# Logging is often deployment specific, but some handlers and -# formatters can be defined here. - -[logging] -[[formatters]] -[[[message_only]]] -format='*(message)s' - -[[[full_content]]] -format='*(asctime)s *(name)s *(levelname)s *(message)s' - -[[handlers]] -[[[debug_out]]] -class='StreamHandler' -level='DEBUG' -args='(sys.stdout,)' -formatter='full_content' - -[[[access_out]]] -class='StreamHandler' -level='INFO' -args='(sys.stdout,)' -formatter='message_only' - -[[[error_out]]] -class='StreamHandler' -level='ERROR' -args='(sys.stdout,)'
deleted file mode 100644 --- a/examples/turbogears/genshitest/controllers.py +++ /dev/null @@ -1,57 +0,0 @@ -import logging - -import cherrypy - -import turbogears -from turbogears import controllers, expose, validate, redirect, widgets -from turbogears import identity - -from genshitest import json - -log = logging.getLogger("genshitest.controllers") - -class Root(controllers.RootController): - - @expose(template="genshitest.templates.welcome") - def index(self): - import time - log.debug("Happy TurboGears Controller Responding For Duty") - return dict(now=time.ctime(), - widget=widgets.TextArea(name="widget_test", - default="Lorem ipsum", - rows=5, cols=40)) - - @expose(template="genshi-text:genshitest.templates.plain", - content_type='text/plain; charset=utf-8') - def plain(self): - return dict(name='world') - - @expose(template="genshitest.templates.login") - def login(self, forward_url=None, previous_url=None, *args, **kw): - - if not identity.current.anonymous \ - and identity.was_login_attempted() \ - and not identity.get_identity_errors(): - raise redirect(forward_url) - - forward_url=None - previous_url= cherrypy.request.path - - if identity.was_login_attempted(): - msg=_("The credentials you supplied were not correct or " - "did not grant access to this resource.") - elif identity.get_identity_errors(): - msg=_("You must provide your credentials before accessing " - "this resource.") - else: - msg=_("Please log in.") - forward_url= cherrypy.request.headers.get("Referer", "/") - cherrypy.response.status=403 - return dict(message=msg, previous_url=previous_url, logging_in=True, - original_parameters=cherrypy.request.params, - forward_url=forward_url) - - @expose() - def logout(self): - identity.current.logout() - raise redirect("/")
deleted file mode 100644 --- a/examples/turbogears/genshitest/json.py +++ /dev/null @@ -1,13 +0,0 @@ -# This module provides helper functions for the JSON part of your -# view, if you are providing a JSON-based API for your app. - -# Here's what most rules would look like: -# @jsonify.when("isinstance(obj, YourClass)") -# def jsonify_yourclass(obj): -# return [obj.val1, obj.val2] -# -# The goal is to break your objects down into simple values: -# lists, dicts, numbers and strings - -from turbojson.jsonify import jsonify -
deleted file mode 100644 --- a/examples/turbogears/genshitest/model.py +++ /dev/null @@ -1,109 +0,0 @@ -from datetime import datetime - -from sqlobject import * - -from turbogears import identity -from turbogears.database import PackageHub - -hub = PackageHub("genshitest") -__connection__ = hub - -# class YourDataClass(SQLObject): -# pass - -class Visit(SQLObject): - class sqlmeta: - table="visit" - - visit_key= StringCol( length=40, alternateID=True, - alternateMethodName="by_visit_key" ) - created= DateTimeCol( default=datetime.now ) - expiry= DateTimeCol() - - def lookup_visit( cls, visit_key ): - try: - return cls.by_visit_key( visit_key ) - except SQLObjectNotFound: - return None - lookup_visit= classmethod(lookup_visit) - -class VisitIdentity(SQLObject): - visit_key = StringCol(length=40, alternateID=True, - alternateMethodName="by_visit_key") - user_id = IntCol() - - -class Group(SQLObject): - """ - An ultra-simple group definition. - """ - - # names like "Group", "Order" and "User" are reserved words in SQL - # so we set the name to something safe for SQL - class sqlmeta: - table="tg_group" - - group_name = UnicodeCol(length=16, alternateID=True, - alternateMethodName="by_group_name") - display_name = UnicodeCol(length=255) - created = DateTimeCol(default=datetime.now) - - # collection of all users belonging to this group - users = RelatedJoin("User", intermediateTable="user_group", - joinColumn="group_id", otherColumn="user_id") - - # collection of all permissions for this group - permissions = RelatedJoin("Permission", joinColumn="group_id", - intermediateTable="group_permission", - otherColumn="permission_id") - - -class User(SQLObject): - """ - Reasonably basic User definition. Probably would want additional attributes. - """ - # names like "Group", "Order" and "User" are reserved words in SQL - # so we set the name to something safe for SQL - class sqlmeta: - table="tg_user" - - user_name = UnicodeCol(length=16, alternateID=True, - alternateMethodName="by_user_name") - email_address = UnicodeCol(length=255, alternateID=True, - alternateMethodName="by_email_address") - display_name = UnicodeCol(length=255) - password = UnicodeCol(length=40) - created = DateTimeCol(default=datetime.now) - - # groups this user belongs to - groups = RelatedJoin("Group", intermediateTable="user_group", - joinColumn="user_id", otherColumn="group_id") - - def _get_permissions(self): - perms = set() - for g in self.groups: - perms = perms | set(g.permissions) - return perms - - def _set_password(self, cleartext_password): - "Runs cleartext_password through the hash algorithm before saving." - hash = identity.encrypt_password(cleartext_password) - self._SO_set_password(hash) - - def set_password_raw(self, password): - "Saves the password as-is to the database." - self._SO_set_password(password) - - - -class Permission(SQLObject): - permission_name = UnicodeCol(length=16, alternateID=True, - alternateMethodName="by_permission_name") - description = UnicodeCol(length=255) - - groups = RelatedJoin("Group", - intermediateTable="group_permission", - joinColumn="permission_id", - otherColumn="group_id") - -
deleted file mode 100644 --- a/examples/turbogears/genshitest/release.py +++ /dev/null @@ -1,14 +0,0 @@ -# Release information about GenshiTest - -version = "1.0" - -# description = "Your plan to rule the world" -# long_description = "More description about your plan" -# author = "Your Name Here" -# email = "YourEmail@YourDomain" -# copyright = "Vintage 2006 - a good year indeed" - -# if it's open source, you might want to specify these -# url = "http://yourcool.site/" -# download_url = "http://yourcool.site/download" -# license = "MIT"
deleted file mode 100644 index 332557bc307647601389c14939be0671c62efcd7..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch literal 0 Hc$@<O00001
deleted file mode 100644 index bc9c79cc6141a67a21edaffc9a068ebd0899eb46..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 GIT binary patch literal 0 Hc$@<O00001
deleted file mode 100644 --- a/examples/turbogears/genshitest/templates/login.html +++ /dev/null @@ -1,115 +0,0 @@ -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" - "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" - xmlns:py="http://genshi.edgewall.org/" - xmlns:xi="http://www.w3.org/2001/XInclude"> - -<xi:include href="master.html" /> - -<head> - <meta content="text/html; charset=UTF-8" - http-equiv="content-type" py:replace="''"/> - <title>Login</title> - <style type="text/css"> - #loginBox - { - width: 30%; - margin: auto; - margin-top: 10%; - padding-left: 10%; - padding-right: 10%; - padding-top: 5%; - padding-bottom: 5%; - font-family: verdana; - font-size: 10px; - background-color: #eee; - border: 2px solid #ccc; - } - - #loginBox h1 - { - font-size: 42px; - font-family: "Trebuchet MS"; - margin: 0; - color: #ddd; - } - - #loginBox p - { - position: relative; - top: -1.5em; - padding-left: 4em; - font-size: 12px; - margin: 0; - color: #666; - } - - #loginBox table - { - table-layout: fixed; - border-spacing: 0; - width: 100%; - } - - #loginBox td.label - { - width: 33%; - text-align: right; - } - - #loginBox td.field - { - width: 66%; - } - - #loginBox td.field input - { - width: 100%; - } - - #loginBox td.buttons - { - text-align: right; - } - - </style> -</head> - -<body> - <div id="loginBox"> - <h1>Login</h1> - <p>$message</p> - <form action="$previous_url" method="POST"> - <table> - <tr> - <td class="label"> - <label for="user_name">User Name:</label> - </td> - <td class="field"> - <input type="text" id="user_name" name="user_name"/> - </td> - </tr> - <tr> - <td class="label"> - <label for="password">Password:</label> - </td> - <td class="field"> - <input type="password" id="password" name="password"/> - </td> - </tr> - <tr> - <td colspan="2" class="buttons"> - <input type="submit" name="login" value="Login"/> - </td> - </tr> - </table> - - <input py:if="forward_url" type="hidden" name="forward_url" - value="$forward_url"/> - - <input py:for="name,value in original_parameters.items()" - type="hidden" name="$name" value="$value"/> - </form> - </div> -</body> -</html>
deleted file mode 100644 --- a/examples/turbogears/genshitest/templates/master.html +++ /dev/null @@ -1,36 +0,0 @@ -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" - "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" - xmlns:xi="http://www.w3.org/2001/XInclude" - xmlns:py="http://genshi.edgewall.org/" - py:strip=""> - <xi:include href="sitetemplate.html"><xi:fallback/></xi:include> - - <head py:match="head" py:attrs="select('@*')"> - <meta content="text/html; charset=UTF-8" http-equiv="content-type" - py:replace="''" /> - <title py:replace="''">Your title goes here</title> - <meta py:replace="select('*')" /> - <style type="text/css"> - #pageLogin { font-size: 10px; font-family: verdana; text-align: right; } - </style> - </head> - - <body py:match="body" py:attrs="select('@*')"> - <div id="pageLogin" - py:if="tg.config('identity.on', False) and not value_of('logging_in')" - py:choose=""> - <span py:when="tg.identity.anonymous"> - <a href="/login">Login</a> - </span> - <span py:otherwise=""> - Welcome ${tg.identity.user.display_name}. - <a href="/logout">Logout</a> - </span> - </div> - <div py:if="tg_flash" class="flash" py:content="tg_flash"></div> - <div py:replace="select('*|text()')" /> - <p align="center"><img src="/static/images/tg_under_the_hood.png" alt="TurboGears under the hood"/></p> - </body> - -</html>
deleted file mode 100644 --- a/examples/turbogears/genshitest/templates/plain.txt +++ /dev/null @@ -1,8 +0,0 @@ -#choose - #when name - Hello, $name! - #end - #otherwise - Hello, anonymous! - #end -#end
deleted file mode 100644 --- a/examples/turbogears/genshitest/templates/sitetemplate.html +++ /dev/null @@ -1,18 +0,0 @@ -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" - xmlns:py="http://genshi.edgewall.org/" - xmlns:xi="http://www.w3.org/2001/XInclude" - py:strip=""> -<head py:match="head" py:attrs="select('@*')"> - <meta content="text/html; charset=UTF-8" http-equiv="content-type" py:replace="''"/> - <title py:replace="''">Your title goes here</title> - <link py:for="css in tg_css" py:replace="ET(css.display())" /> - <link py:for="js in tg_js_head" py:replace="ET(js.display())" /> - <meta py:replace="select('*')" /> -</head> -<body py:match="body" py:attrs="select('@*')"> - <div py:for="js in tg_js_bodytop" py:replace="ET(js.display())" /> - <div py:replace="select('*|text()')" /> - <div py:for="js in tg_js_bodybottom" py:replace="ET(js.display())" /> -</body> -</html>
deleted file mode 100644 --- a/examples/turbogears/genshitest/templates/welcome.html +++ /dev/null @@ -1,34 +0,0 @@ -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" - "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" - xmlns:xi="http://www.w3.org/2001/XInclude" - xmlns:py="http://genshi.edgewall.org/"> - <xi:include href="master.html" /> - - <head> - <meta content="text/html; charset=UTF-8" http-equiv="content-type" py:replace="''"/> - <title>Welcome to TurboGears</title> - </head> - - <body> - <p>Congratulations, your TurboGears application is running as of - <span py:replace="now">now</span>.</p> - - <h2>Using Genshi in TurboGears</h2> - - <p> - Please see the online - <a href="http://genshi.edgewall.org/wiki/Documentation">documentation</a> - for general information on Genshi. - </p> - - <p>Here's an example for using a TurboGears widget in a Genshi template:</p> - ${ET(widget.display())} - - <p> - And here's a <a href="plain">link</a> to the output of a plain-text - template. - </p> - - </body> -</html>
deleted file mode 100644 --- a/examples/turbogears/genshitest/tests/test_controllers.py +++ /dev/null @@ -1,16 +0,0 @@ -from turbogears import testutil -from genshitest.controllers import Root -import cherrypy - -cherrypy.root = Root() - -def test_method(): - "the index method should return a string called now" - import types - result = testutil.call(cherrypy.root.index) - assert type(result["now"]) == types.StringType - -def test_indextitle(): - "The mainpage should have the right title" - testutil.createRequest("/") - assert "<TITLE>Welcome to TurboGears</TITLE>" in cherrypy.response.body[0]
deleted file mode 100644 --- a/examples/turbogears/genshitest/tests/test_model.py +++ /dev/null @@ -1,23 +0,0 @@ -# If your project uses a database, you can set up database tests -# similar to what you see below. Be sure to set the db_uri to -# an appropriate uri for your testing database. sqlite is a good -# choice for testing, because you can use an in-memory database -# which is very fast. - -from turbogears import testutil -# from genshitest.model import YourDataClass, User - -# database.set_db_uri("sqlite:///:memory:") - -# class TestUser(testutil.DBTest): -# def get_model(self): -# return User -# -# def test_creation(self): -# "Object creation should set the name" -# obj = User(user_name = "creosote", -# email_address = "spam@python.not", -# display_name = "Mr Creosote", -# password = "Wafer-thin Mint") -# assert obj.display_name == "Mr Creosote" -
deleted file mode 100644 --- a/examples/turbogears/sample-prod.cfg +++ /dev/null @@ -1,69 +0,0 @@ -[global] -# This is where all of your settings go for your production environment. -# You'll copy this file over to your production server and provide it -# as a command-line option to your start script. -# Settings that are the same for both development and production -# (such as template engine, encodings, etc.) all go in -# genshitest/config/app.cfg - -# pick the form for your database -# sqlobject.dburi="postgres://username@hostname/databasename" -# sqlobject.dburi="mysql://username:password@hostname:port/databasename" -# sqlobject.dburi="sqlite:///file_name_and_path" - -# If you have sqlite, here's a simple default to get you started -# in development -sqlobject.dburi="sqlite://%(current_dir_uri)s/devdata.sqlite" - - -# if you are using a database or table type without transactions -# (MySQL default, for example), you should turn off transactions -# by prepending notrans_ on the uri -# sqlobject.dburi="notrans_mysql://username:password@hostname:port/databasename" - -# for Windows users, sqlite URIs look like: -# sqlobject.dburi="sqlite:///drive_letter:/path/to/file" - - -# SERVER - -server.environment="production" - -# Sets the number of threads the server uses -# server.thread_pool = 1 - -# if this is part of a larger site, you can set the path -# to the TurboGears instance here -# server.webpath="" - -# Set to True if you'd like to abort execution if a controller gets an -# unexpected parameter. False by default -# tg.strict_parameters = False - -# LOGGING -# Logging configuration generally follows the style of the standard -# Python logging module configuration. Note that when specifying -# log format messages, you need to use *() for formatting variables. -# Deployment independent log configuration is in genshitest/config/log.cfg -[logging] - -[[handlers]] - -[[[access_out]]] -# set the filename as the first argument below -args="('server.log',)" -class='FileHandler' -level='INFO' -formatter='message_only' - -[[loggers]] -[[[genshitest]]] -level='ERROR' -qualname='genshitest' -handlers=['error_out'] - -[[[access]]] -level='INFO' -qualname='turbogears.access' -handlers=['access_out'] -propagate=0
deleted file mode 100644 --- a/examples/turbogears/setup.py +++ /dev/null @@ -1,62 +0,0 @@ -from setuptools import setup, find_packages -from turbogears.finddata import find_package_data - -import os -execfile(os.path.join("genshitest", "release.py")) - -setup( - name="GenshiTest", - version=version, - - # uncomment the following lines if you fill them out in release.py - #description=description, - #author=author, - #author_email=email, - #url=url, - #download_url=download_url, - #license=license, - - install_requires = [ - "TurboGears >= 0.9a9dev-r1686", - ], - scripts = ["start-genshitest.py"], - zip_safe=False, - packages=find_packages(), - package_data = find_package_data(where='genshitest', - package='genshitest'), - keywords = [ - # Use keywords if you'll be adding your package to the - # Python Cheeseshop - - # if this has widgets, uncomment the next line - # 'turbogears.widgets', - - # if this has a tg-admin command, uncomment the next line - # 'turbogears.command', - - # if this has identity providers, uncomment the next line - # 'turbogears.identity.provider', - - # If this is a template plugin, uncomment the next line - # 'python.templating.engines', - - # If this is a full application, uncomment the next line - # 'turbogears.app', - ], - classifiers = [ - 'Development Status :: 3 - Alpha', - 'Operating System :: OS Independent', - 'Programming Language :: Python', - 'Topic :: Software Development :: Libraries :: Python Modules', - 'Framework :: TurboGears', - # if this is an application that you'll distribute through - # the Cheeseshop, uncomment the next line - # 'Framework :: TurboGears :: Applications', - - # if this is a package that includes widgets that you'll distribute - # through the Cheeseshop, uncomment the next line - # 'Framework :: TurboGears :: Widgets', - ], - test_suite = 'nose.collector', - ) -
deleted file mode 100755 --- a/examples/turbogears/start-genshitest.py +++ /dev/null @@ -1,28 +0,0 @@ -#!/usr/bin/python -import pkg_resources -pkg_resources.require("TurboGears") - -import turbogears -import cherrypy -cherrypy.lowercase_api = True - -from os.path import * -import sys - -# first look on the command line for a desired config file, -# if it's not on the command line, then -# look for setup.py in this directory. If it's not there, this script is -# probably installed -if len(sys.argv) > 1: - turbogears.update_config(configfile=sys.argv[1], - modulename="genshitest.config") -elif exists(join(dirname(__file__), "setup.py")): - turbogears.update_config(configfile="dev.cfg", - modulename="genshitest.config") -else: - turbogears.update_config(configfile="prod.cfg", - modulename="genshitest.config") - -from genshitest.controllers import Root - -turbogears.start_server(Root())
new file mode 100755 --- /dev/null +++ b/examples/tutorial/geddit/controller.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python + +import operator, os, pickle, sys + +import cherrypy +from formencode import Invalid +from genshi.input import HTML +from genshi.filters import HTMLFormFiller, HTMLSanitizer + +from geddit.form import LinkForm, CommentForm +from geddit.lib import ajax, template +from geddit.model import Link, Comment + + +class Root(object): + + def __init__(self, data): + self.data = data + + @cherrypy.expose + @template.output('index.html') + def index(self): + links = sorted(self.data.values(), key=operator.attrgetter('time')) + return template.render(links=links) + + @cherrypy.expose + @template.output('submit.html') + def submit(self, cancel=False, **data): + if cherrypy.request.method == 'POST': + if cancel: + raise cherrypy.HTTPRedirect('/') + form = LinkForm() + try: + data = form.to_python(data) + link = Link(**data) + self.data[link.id] = link + raise cherrypy.HTTPRedirect('/') + except Invalid, e: + errors = e.unpack_errors() + else: + errors = {} + + return template.render(errors=errors) | HTMLFormFiller(data=data) + + @cherrypy.expose + @template.output('info.html') + def info(self, id): + link = self.data.get(id) + if not link: + raise cherrypy.NotFound() + return template.render(link=link) + + @cherrypy.expose + @template.output('comment.html') + def comment(self, id, cancel=False, **data): + link = self.data.get(id) + if not link: + raise cherrypy.NotFound() + if cherrypy.request.method == 'POST': + if cancel: + raise cherrypy.HTTPRedirect('/info/%s' % link.id) + form = CommentForm() + try: + data = form.to_python(data) + markup = HTML(data['content']) | HTMLSanitizer() + data['content'] = markup.render('xhtml') + comment = link.add_comment(**data) + if not ajax.is_xhr(): + raise cherrypy.HTTPRedirect('/info/%s' % link.id) + return template.render('_comment.html', comment=comment, + num=len(link.comments)) + except Invalid, e: + errors = e.unpack_errors() + else: + errors = {} + + if ajax.is_xhr(): + stream = template.render('_form.html', link=link, errors=errors) + else: + stream = template.render(link=link, comment=None, errors=errors) + return stream | HTMLFormFiller(data=data) + + @cherrypy.expose + @template.output('index.xml', method='xml') + def feed(self, id=None): + if id: + link = self.data.get(id) + if not link: + raise cherrypy.NotFound() + return template.render('info.xml', link=link) + else: + links = sorted(self.data.values(), key=operator.attrgetter('time')) + return template.render(links=links) + + +def main(filename): + # load data from the pickle file, or initialize it to an empty list + if os.path.exists(filename): + fileobj = open(filename, 'rb') + try: + data = pickle.load(fileobj) + finally: + fileobj.close() + else: + data = {} + + def _save_data(): + # save data back to the pickle file + fileobj = open(filename, 'wb') + try: + pickle.dump(data, fileobj) + finally: + fileobj.close() + if hasattr(cherrypy.engine, 'subscribe'): # CherryPy >= 3.1 + cherrypy.engine.subscribe('stop', _save_data) + else: + cherrypy.engine.on_stop_engine_list.append(_save_data) + + # Some global configuration; note that this could be moved into a + # configuration file + cherrypy.config.update({ + 'tools.encode.on': True, 'tools.encode.encoding': 'utf-8', + 'tools.decode.on': True, + 'tools.trailing_slash.on': True, + 'tools.staticdir.root': os.path.abspath(os.path.dirname(__file__)), + }) + + cherrypy.quickstart(Root(data), '/', { + '/media': { + 'tools.staticdir.on': True, + 'tools.staticdir.dir': 'static' + } + }) + +if __name__ == '__main__': + import formencode + formencode.api.set_stdtranslation(languages=['en']) + main(sys.argv[1])
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/form.py @@ -0,0 +1,12 @@ +from formencode import Schema, validators + + +class LinkForm(Schema): + username = validators.UnicodeString(not_empty=True) + url = validators.URL(not_empty=True, add_http=True, check_exists=False) + title = validators.UnicodeString(not_empty=True) + + +class CommentForm(Schema): + username = validators.UnicodeString(not_empty=True) + content = validators.UnicodeString(not_empty=True)
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/lib/ajax.py @@ -0,0 +1,5 @@ +import cherrypy + +def is_xhr(): + requested_with = cherrypy.request.headers.get('X-Requested-With') + return requested_with and requested_with.lower() == 'xmlhttprequest'
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/lib/template.py @@ -0,0 +1,47 @@ +import os + +import cherrypy +from genshi.core import Stream +from genshi.output import encode, get_serializer +from genshi.template import Context, TemplateLoader + +from geddit.lib import ajax + +loader = TemplateLoader( + os.path.join(os.path.dirname(__file__), '..', 'templates'), + auto_reload=True +) + +def output(filename, method='html', encoding='utf-8', **options): + """Decorator for exposed methods to specify what template the should use + for rendering, and which serialization method and options should be + applied. + """ + def decorate(func): + def wrapper(*args, **kwargs): + cherrypy.thread_data.template = loader.load(filename) + opt = options.copy() + if not ajax.is_xhr() and method == 'html': + opt.setdefault('doctype', 'html') + serializer = get_serializer(method, **opt) + stream = func(*args, **kwargs) + if not isinstance(stream, Stream): + return stream + return encode(serializer(stream), method=serializer, + encoding=encoding) + return wrapper + return decorate + +def render(*args, **kwargs): + """Function to render the given data to the template specified via the + ``@output`` decorator. + """ + if args: + assert len(args) == 1, \ + 'Expected exactly one argument, but got %r' % (args,) + template = loader.load(args[0]) + else: + template = cherrypy.thread_data.template + ctxt = Context(url=cherrypy.url) + ctxt.push(kwargs) + return template.generate(ctxt)
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/model.py @@ -0,0 +1,31 @@ +from datetime import datetime + + +class Link(object): + + def __init__(self, username, url, title): + self.username = username + self.url = url + self.title = title + self.time = datetime.utcnow() + self.id = hex(hash(tuple([username, url, title, self.time])))[2:] + self.comments = [] + + def __repr__(self): + return '<%s %r>' % (type(self).__name__, self.title) + + def add_comment(self, username, content): + comment = Comment(username, content) + self.comments.append(comment) + return comment + + +class Comment(object): + + def __init__(self, username, content): + self.username = username + self.content = content + self.time = datetime.utcnow() + + def __repr__(self): + return '<%s by %r>' % (type(self).__name__, self.username)
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/static/jquery.js @@ -0,0 +1,11 @@ +/* + * jQuery 1.1.4 - New Wave Javascript + * + * Copyright (c) 2007 John Resig (jquery.com) + * Dual licensed under the MIT (MIT-LICENSE.txt) + * and GPL (GPL-LICENSE.txt) licenses. + * + * $Date: 2007-08-23 21:49:27 -0400 (Thu, 23 Aug 2007) $ + * $Rev: 2862 $ + */ +eval(function(p,a,c,k,e,r){e=function(c){return(c<a?'':e(parseInt(c/a)))+((c=c%a)>35?String.fromCharCode(c+29):c.toString(36))};if(!''.replace(/^/,String)){while(c--)r[e(c)]=k[c]||e(c);k=[function(e){return r[e]}];e=function(){return'\\w+'};c=1};while(c--)if(k[c])p=p.replace(new RegExp('\\b'+e(c)+'\\b','g'),k[c]);return p}('(9(){6(1f C!="Q")E v=C;E C=19.16=9(a,c){6(19==7||!7.4a)F 1s C(a,c);F 7.4a(a,c)};6(1f $!="Q")E B=$;19.$=C;E q=/^[^<]*(<(.|\\s)+>)[^>]*$|^#(\\w+)$/;C.15=C.3v={4a:9(a,c){a=a||R;6(1f a=="1E"){E m=q.2d(a);6(m&&(m[1]||!c)){6(m[1])a=C.3c([m[1]]);G{E b=R.37(m[3]);6(b)6(b.2j!=m[3])F C().1F(a);G{7[0]=b;7.H=1;F 7}G a=[]}}G F 1s C(c).1F(a)}G 6(C.1g(a))F 1s C(R)[C.15.1L?"1L":"2f"](a);F 7.5J(a.1b==1K&&a||(a.3w||a.H&&a!=19&&!a.1t&&a[0]!=Q&&a[0].1t)&&C.2V(a)||[a])},3w:"1.1.4",7K:9(){F 7.H},H:0,21:9(a){F a==Q?C.2V(7):7[a]},1O:9(a){E b=C(a);b.5c=7;F b},5J:9(a){7.H=0;1K.3v.Y.T(7,a);F 7},J:9(a,b){F C.J(7,a,b)},45:9(a){E b=-1;7.J(9(i){6(7==a)b=i});F b},1j:9(f,d,e){E c=f;6(f.1b==3n)6(d==Q)F 7.H&&C[e||"1j"](7[0],f)||Q;G{c={};c[f]=d}F 7.J(9(a){I(E b 17 c)C.1j(e?7.S:7,b,C.4Q(7,c[b],e,a,b))})},1h:9(b,a){F 7.1j(b,a,"34")},2Q:9(e){6(1f e!="4P"&&e!=K)F 7.3K().3H(R.60(e));E t="";C.J(e||7,9(){C.J(7.2Z,9(){6(7.1t!=8)t+=7.1t!=1?7.5S:C.15.2Q([7])})});F t},82:9(){E a,2e=1a;F 7.J(9(){6(!a)a=C.3c(2e,7.2I);E b=a[0].3B(O);7.P.2p(b,7);20(b.1k)b=b.1k;b.4p(7)})},3H:9(){F 7.2J(1a,O,1,9(a){7.4p(a)})},5v:9(){F 7.2J(1a,O,-1,9(a){7.2p(a,7.1k)})},5u:9(){F 7.2J(1a,M,1,9(a){7.P.2p(a,7)})},5t:9(){F 7.2J(1a,M,-1,9(a){7.P.2p(a,7.2a)})},3L:9(){F 7.5c||C([])},1F:9(t){E b=C.3M(7,9(a){F C.1F(t,a)});F 7.1O(/[^+>] [^+>]/.1d(t)||t.U("..")>-1?C.4d(b):b)},7o:9(e){e=e!=Q?e:O;E d=7.1r(7.1F("*"));6(C.N.12){d.J(9(){7.2l$1i={};I(E a 17 7.$1i)7.2l$1i[a]=C.14({},7.$1i[a])}).49()}E r=7.1O(C.3M(7,9(a){F a.3B(e)}));6(C.N.12){d.J(9(){E c=7.2l$1i;I(E a 17 c)I(E b 17 c[a])C.1c.1r(7,a,c[a][b],c[a][b].V);7.2l$1i=K})}6(e){E f=r.1r(r.1F(\'*\')).1l(\'2b,39[@L=3i]\');d.1l(\'2b,39[@L=3i]\').J(9(i){6(7.3j)f[i].3j=7.3j;6(7.27)f[i].27=O})}F r},1l:9(t){F 7.1O(C.1g(t)&&C.2B(7,9(b,a){F t.T(b,[a])})||C.2R(t,7))},5l:9(t){F 7.1O(t.1b==3n&&C.2R(t,7,O)||C.2B(7,9(a){F(t.1b==1K||t.3w)?C.4K(a,t)<0:a!=t}))},1r:9(t){F 7.1O(C.29(7.21(),t.1b==3n?C(t).21():t.H!=Q&&(!t.W||t.W=="6s")?t:[t]))},3y:9(a){F a?C.2R(a,7).H>0:M},2G:9(a){F a==Q?(7.H?7[0].2A:K):7.1j("2A",a)},5W:9(a){F a==Q?(7.H?7[0].2W:K):7.3K().3H(a)},3S:9(){F 7.1O(1K.3v.3S.T(7,1a))},2J:9(f,d,g,e){E c=7.H>1,a;F 7.J(9(){6(!a){a=C.3c(f,7.2I);6(g<0)a.8E()}E b=7;6(d&&C.W(7,"1A")&&C.W(a[0],"3O"))b=7.4L("1w")[0]||7.4p(R.6a("1w"));C.J(a,9(){6(C.W(7,"33")){6(7.32)C.31({1G:7.32,2w:M,3G:"33"});G C.4E(7.2Q||7.5Z||7.2W||"")}G e.T(b,[c?7.3B(O):7])})})}};C.14=C.15.14=9(){E c=1a[0]||{},a=1,1M=1a.H,4D=M;6(c.1b==8d){4D=c;c=1a[1]||{}}6(1M==1){c=7;a=0}E b;I(;a<1M;a++)6((b=1a[a])!=K)I(E i 17 b){6(c==b[i])5X;6(4D&&1f b[i]==\'4P\'&&c[i])C.14(c[i],b[i]);G 6(b[i]!=Q)c[i]=b[i]}F c};C.14({8a:9(a){19.$=B;6(a)19.16=v;F C},1g:9(a){F!!a&&1f a!="1E"&&!a.W&&a.1b!=1K&&/9/i.1d(a+"")},3E:9(a){F a.3D&&!a.4z||a.4y&&a.2I&&!a.2I.4z},4E:9(a){a=C.2s(a);6(a){6(19.5N)19.5N(a);G 6(C.N.1H)19.4x(a,0);G 2T.2S(19,a)}},W:9(b,a){F b.W&&b.W.1I()==a.1I()},J:9(a,b,c){6(c){6(a.H==Q)I(E i 17 a)b.T(a[i],c);G I(E i=0,3A=a.H;i<3A;i++)6(b.T(a[i],c)===M)1J}G{6(a.H==Q)I(E i 17 a)b.2S(a[i],i,a[i]);G I(E i=0,3A=a.H,2G=a[0];i<3A&&b.2S(2G,i,2G)!==M;2G=a[++i]){}}F a},4Q:9(c,b,d,e,a){6(C.1g(b))b=b.2S(c,[e]);E f=/z-?45|7S-?7Q|1e|5y|7O-?1u/i;F b&&b.1b==3x&&d=="34"&&!f.1d(a)?b+"4t":b},18:{1r:9(b,c){C.J((c||"").2M(/\\s+/),9(i,a){6(!C.18.2N(b.18,a))b.18+=(b.18?" ":"")+a})},23:9(b,c){b.18=c!=Q?C.2B(b.18.2M(/\\s+/),9(a){F!C.18.2N(c,a)}).5w(" "):""},2N:9(t,c){F C.4K(c,(t.18||t).3s().2M(/\\s+/))>-1}},1V:9(e,o,f){I(E i 17 o){e.S["2U"+i]=e.S[i];e.S[i]=o[i]}f.T(e,[]);I(E i 17 o)e.S[i]=e.S["2U"+i]},1h:9(e,p){6(p=="1u"||p=="24"){E b={},3p,3o,d=["7J","7G","7F","7B"];C.J(d,9(){b["7A"+7]=0;b["7x"+7+"7u"]=0});C.1V(e,b,9(){6(C(e).3y(\':4N\')){3p=e.7t;3o=e.7q}G{e=C(e.3B(O)).1F(":4e").5d("27").3L().1h({3V:"1C",3k:"7n",11:"2m",7h:"0",7e:"0"}).57(e.P)[0];E a=C.1h(e.P,"3k")||"3g";6(a=="3g")e.P.S.3k="76";3p=e.74;3o=e.71;6(a=="3g")e.P.S.3k="3g";e.P.3e(e)}});F p=="1u"?3p:3o}F C.34(e,p)},34:9(h,d,g){E i,1R=[],1V=[];9 2E(a){6(!C.N.1H)F M;E b=R.2L.3b(a,K);F!b||b.44("2E")==""}6(d=="1e"&&C.N.12){i=C.1j(h.S,"1e");F i==""?"1":i}6(d.2k(/3a/i))d=x;6(!g&&h.S[d])i=h.S[d];G 6(R.2L&&R.2L.3b){6(d.2k(/3a/i))d="3a";d=d.1v(/([A-Z])/g,"-$1").2D();E e=R.2L.3b(h,K);6(e&&!2E(h))i=e.44(d);G{I(E a=h;a&&2E(a);a=a.P)1R.42(a);I(a=0;a<1R.H;a++)6(2E(1R[a])){1V[a]=1R[a].S.11;1R[a].S.11="2m"}i=d=="11"&&1V[1R.H-1]!=K?"1T":R.2L.3b(h,K).44(d)||"";I(a=0;a<1V.H;a++)6(1V[a]!=K)1R[a].S.11=1V[a]}6(d=="1e"&&i=="")i="1"}G 6(h.41){E f=d.1v(/\\-(\\w)/g,9(m,c){F c.1I()});i=h.41[d]||h.41[f]}F i},3c:9(a,c){E r=[];c=c||R;C.J(a,9(i,b){6(!b)F;6(b.1b==3x)b=b.3s();6(1f b=="1E"){E s=C.2s(b).2D(),1m=c.6a("1m"),1P=[];E a=!s.U("<1Z")&&[1,"<2b>","</2b>"]||!s.U("<6L")&&[1,"<4V>","</4V>"]||s.2k(/^<(6I|1w|6H|6F|6D)/)&&[1,"<1A>","</1A>"]||!s.U("<3O")&&[2,"<1A><1w>","</1w></1A>"]||(!s.U("<6A")||!s.U("<6y"))&&[3,"<1A><1w><3O>","</3O></1w></1A>"]||!s.U("<6x")&&[2,"<1A><1w></1w><4T>","</4T></1A>"]||C.N.12&&[1,"1m<1m>","</1m>"]||[0,"",""];1m.2W=a[1]+b+a[2];20(a[0]--)1m=1m.3Y;6(C.N.12){6(!s.U("<1A")&&s.U("<1w")<0)1P=1m.1k&&1m.1k.2Z;G 6(a[1]=="<1A>"&&s.U("<1w")<0)1P=1m.2Z;I(E n=1P.H-1;n>=0;--n)6(C.W(1P[n],"1w")&&!1P[n].2Z.H)1P[n].P.3e(1P[n]);6(/^\\s/.1d(b))1m.2p(c.60(b.2k(/^\\s*/)[0]),1m.1k)}b=C.2V(1m.2Z)}6(0===b.H&&(!C.W(b,"38")&&!C.W(b,"2b")))F;6(b[0]==Q||C.W(b,"38")||b.6u)r.Y(b);G r=C.29(r,b)});F r},1j:9(c,d,a){E e=C.3E(c)?{}:C.4q;6(d=="28"&&C.N.1H)c.P.3j;6(e[d]){6(a!=Q)c[e[d]]=a;F c[e[d]]}G 6(C.N.12&&d=="S")F C.1j(c.S,"6p",a);G 6(a==Q&&C.N.12&&C.W(c,"38")&&(d=="6n"||d=="6m"))F c.6k(d).5S;G 6(c.4y){6(a!=Q)c.6j(d,a);6(C.N.12&&/5R|32/.1d(d)&&!C.3E(c))F c.3F(d,2);F c.3F(d)}G{6(d=="1e"&&C.N.12){6(a!=Q){c.5y=1;c.1l=(c.1l||"").1v(/5T\\([^)]*\\)/,"")+(3m(a).3s()=="6d"?"":"5T(1e="+a*6c+")")}F c.1l?(3m(c.1l.2k(/1e=([^)]*)/)[1])/6c).3s():""}d=d.1v(/-([a-z])/8I,9(z,b){F b.1I()});6(a!=Q)c[d]=a;F c[d]}},2s:9(t){F(t||"").1v(/^\\s+|\\s+$/g,"")},2V:9(a){E r=[];6(1f a!="8H")I(E i=0,1M=a.H;i<1M;i++)r.Y(a[i]);G r=a.3S(0);F r},4K:9(b,a){I(E i=0,1M=a.H;i<1M;i++)6(a[i]==b)F i;F-1},29:9(a,b){6(C.N.12){I(E i=0;b[i];i++)6(b[i].1t!=8)a.Y(b[i])}G I(E i=0;b[i];i++)a.Y(b[i]);F a},4d:9(a){E r=[],4O=C.1q++;2g{I(E i=0,69=a.H;i<69;i++)6(4O!=a[i].1q){a[i].1q=4O;r.Y(a[i])}}2h(e){r=a}F r},1q:0,2B:9(b,a,c){6(1f a=="1E")a=2T("M||9(a,i){F "+a+"}");E d=[];I(E i=0,3P=b.H;i<3P;i++)6(!c&&a(b[i],i)||c&&!a(b[i],i))d.Y(b[i]);F d},3M:9(c,b){6(1f b=="1E")b=2T("M||9(a){F "+b+"}");E d=[];I(E i=0,3P=c.H;i<3P;i++){E a=b(c[i],i);6(a!==K&&a!=Q){6(a.1b!=1K)a=[a];d=d.8x(a)}}F d}});E u=8w.8u.2D();C.N={6b:(u.2k(/.+(?:8s|8q|8p|8o)[\\/: ]([\\d.]+)/)||[])[1],1H:/61/.1d(u),2t:/2t/.1d(u),12:/12/.1d(u)&&!/2t/.1d(u),3J:/3J/.1d(u)&&!/(8n|61)/.1d(u)};E x=C.N.12?"3I":"4G";C.14({8m:!C.N.12||R.8l=="8k",3I:C.N.12?"3I":"4G",4q:{"I":"8j","8i":"18","3a":x,4G:x,3I:x,2W:"2W",18:"18",2A:"2A",30:"30",27:"27",8h:"8g",28:"28",8f:"8e"}});C.J({5Y:"a.P",4C:"16.4C(a)",8c:"16.25(a,2,\'2a\')",8b:"16.25(a,2,\'4B\')",88:"16.4A(a.P.1k,a)",87:"16.4A(a.1k)"},9(i,n){C.15[i]=9(a){E b=C.3M(7,n);6(a&&1f a=="1E")b=C.2R(a,b);F 7.1O(C.4d(b))}});C.J({57:"3H",86:"5v",2p:"5u",85:"5t"},9(i,n){C.15[i]=9(){E a=1a;F 7.J(9(){I(E j=0,1M=a.H;j<1M;j++)C(a[j])[n](7)})}});C.J({5d:9(a){C.1j(7,a,"");7.84(a)},83:9(c){C.18.1r(7,c)},81:9(c){C.18.23(7,c)},80:9(c){C.18[C.18.2N(7,c)?"23":"1r"](7,c)},23:9(a){6(!a||C.1l(a,[7]).r.H)7.P.3e(7)},3K:9(){20(7.1k)7.3e(7.1k)}},9(i,n){C.15[i]=9(){F 7.J(n,1a)}});C.J(["5Q","5P","5M","5L"],9(i,n){C.15[n]=9(a,b){F 7.1l(":"+n+"("+a+")",b)}});C.J(["1u","24"],9(i,n){C.15[n]=9(h){F h==Q?(7.H?C.1h(7[0],n):K):7.1h(n,h.1b==3n?h:h+"4t")}});E A=C.N.1H&&5K(C.N.6b)<7Z?"(?:[\\\\w*2l-]|\\\\\\\\.)":"(?:[\\\\w\\7Y-\\7V*2l-]|\\\\\\\\.)",5I=1s 3C("^[/>]\\\\s*("+A+"+)"),5H=1s 3C("^("+A+"+)(#)("+A+"+)"),5G=1s 3C("^([#.]?)("+A+"*)");C.14({4w:{"":"m[2]==\'*\'||16.W(a,m[2])","#":"a.3F(\'2j\')==m[2]",":":{5P:"i<m[3]-0",5M:"i>m[3]-0",25:"m[3]-0==i",5Q:"m[3]-0==i",2H:"i==0",2P:"i==r.H-1",5E:"i%2==0",5D:"i%2","2H-3z":"a.P.4L(\'*\')[0]==a","2P-3z":"16.25(a.P.3Y,1,\'4B\')==a","7U-3z":"!16.25(a.P.3Y,2,\'4B\')",5Y:"a.1k",3K:"!a.1k",5L:"(a.5Z||a.7T||\'\').U(m[3])>=0",4N:\'"1C"!=a.L&&16.1h(a,"11")!="1T"&&16.1h(a,"3V")!="1C"\',1C:\'"1C"==a.L||16.1h(a,"11")=="1T"||16.1h(a,"3V")=="1C"\',7R:"!a.30",30:"a.30",27:"a.27",28:"a.28||16.1j(a,\'28\')",2Q:"\'2Q\'==a.L",4e:"\'4e\'==a.L",3i:"\'3i\'==a.L",4v:"\'4v\'==a.L",5C:"\'5C\'==a.L",4u:"\'4u\'==a.L",5B:"\'5B\'==a.L",5A:"\'5A\'==a.L",1X:\'"1X"==a.L||16.W(a,"1X")\',39:"/39|2b|7P|1X/i.1d(a.W)",2N:"16.1F(m[3],a).H"},"[":"16.1F(m[2],a).H"},5x:[/^\\[ *(@)([\\w-]+) *([!*$^~=]*) *(\'?"?)(.*?)\\4 *\\]/,/^(\\[)\\s*(.*?(\\[.*?\\])?[^[]*?)\\s*\\]/,/^(:)([\\w-]+)\\("?\'?(.*?(\\(.*?\\))?[^(]*?)"?\'?\\)/,1s 3C("^([:.#]*)("+A+"+)")],2R:9(a,c,b){E d,1Y=[];20(a&&a!=d){d=a;E f=C.1l(a,c,b);a=f.t.1v(/^\\s*,\\s*/,"");1Y=b?c=f.r:C.29(1Y,f.r)}F 1Y},1F:9(t,l){6(1f t!="1E")F[t];6(l&&!l.1t)l=K;l=l||R;6(!t.U("//")){t=t.2K(2,t.H)}G 6(!t.U("/")&&!l.2I){l=l.3D;t=t.2K(1,t.H);6(t.U("/")>=1)t=t.2K(t.U("/"),t.H)}E d=[l],2q=[],2P;20(t&&2P!=t){E r=[];2P=t;t=C.2s(t).1v(/^\\/\\//,"");E k=M;E g=5I;E m=g.2d(t);6(m){E o=m[1].1I();I(E i=0;d[i];i++)I(E c=d[i].1k;c;c=c.2a)6(c.1t==1&&(o=="*"||c.W.1I()==o.1I()))r.Y(c);d=r;t=t.1v(g,"");6(t.U(" ")==0)5X;k=O}G{g=/^((\\/?\\.\\.)|([>\\/+~]))\\s*(\\w*)/i;6((m=g.2d(t))!=K){r=[];E o=m[4],1q=C.1q++;m=m[1];I(E j=0,2o=d.H;j<2o;j++)6(m.U("..")<0){E n=m=="~"||m=="+"?d[j].2a:d[j].1k;I(;n;n=n.2a)6(n.1t==1){6(m=="~"&&n.1q==1q)1J;6(!o||n.W.1I()==o.1I()){6(m=="~")n.1q=1q;r.Y(n)}6(m=="+")1J}}G r.Y(d[j].P);d=r;t=C.2s(t.1v(g,""));k=O}}6(t&&!k){6(!t.U(",")){6(l==d[0])d.4s();2q=C.29(2q,d);r=d=[l];t=" "+t.2K(1,t.H)}G{E h=5H;E m=h.2d(t);6(m){m=[0,m[2],m[3],m[1]]}G{h=5G;m=h.2d(t)}m[2]=m[2].1v(/\\\\/g,"");E f=d[d.H-1];6(m[1]=="#"&&f&&f.37&&!C.3E(f)){E p=f.37(m[2]);6((C.N.12||C.N.2t)&&p&&1f p.2j=="1E"&&p.2j!=m[2])p=C(\'[@2j="\'+m[2]+\'"]\',f)[0];d=r=p&&(!m[3]||C.W(p,m[3]))?[p]:[]}G{I(E i=0;d[i];i++){E a=m[1]!=""||m[0]==""?"*":m[2];6(a=="*"&&d[i].W.2D()=="4P")a="2O";r=C.29(r,d[i].4L(a))}6(m[1]==".")r=C.4r(r,m[2]);6(m[1]=="#"){E e=[];I(E i=0;r[i];i++)6(r[i].3F("2j")==m[2]){e=[r[i]];1J}r=e}d=r}t=t.1v(h,"")}}6(t){E b=C.1l(t,r);d=r=b.r;t=C.2s(b.t)}}6(t)d=[];6(d&&l==d[0])d.4s();2q=C.29(2q,d);F 2q},4r:9(r,m,a){m=" "+m+" ";E c=[];I(E i=0;r[i];i++){E b=(" "+r[i].18+" ").U(m)>=0;6(!a&&b||a&&!b)c.Y(r[i])}F c},1l:9(t,r,h){E d;20(t&&t!=d){d=t;E p=C.5x,m;I(E i=0;p[i];i++){m=p[i].2d(t);6(m){t=t.7N(m[0].H);m[2]=m[2].1v(/\\\\/g,"");1J}}6(!m)1J;6(m[1]==":"&&m[2]=="5l")r=C.1l(m[3],r,O).r;G 6(m[1]==".")r=C.4r(r,m[2],h);G 6(m[1]=="@"){E g=[],L=m[3];I(E i=0,2o=r.H;i<2o;i++){E a=r[i],z=a[C.4q[m[2]]||m[2]];6(z==K||/5R|32|28/.1d(m[2]))z=C.1j(a,m[2])||\'\';6((L==""&&!!z||L=="="&&z==m[5]||L=="!="&&z!=m[5]||L=="^="&&z&&!z.U(m[5])||L=="$="&&z.2K(z.H-m[5].H)==m[5]||(L=="*="||L=="~=")&&z.U(m[5])>=0)^h)g.Y(a)}r=g}G 6(m[1]==":"&&m[2]=="25-3z"){E e=C.1q++,g=[],1d=/(\\d*)n\\+?(\\d*)/.2d(m[3]=="5E"&&"2n"||m[3]=="5D"&&"2n+1"||!/\\D/.1d(m[3])&&"n+"+m[3]||m[3]),2H=(1d[1]||1)-0,d=1d[2]-0;I(E i=0,2o=r.H;i<2o;i++){E j=r[i],P=j.P;6(e!=P.1q){E c=1;I(E n=P.1k;n;n=n.2a)6(n.1t==1)n.4o=c++;P.1q=e}E b=M;6(2H==1){6(d==0||j.4o==d)b=O}G 6((j.4o+d)%2H==0)b=O;6(b^h)g.Y(j)}r=g}G{E f=C.4w[m[1]];6(1f f!="1E")f=C.4w[m[1]][m[2]];f=2T("M||9(a,i){F "+f+"}");r=C.2B(r,f,h)}}F{r:r,t:t}},4C:9(c){E b=[];E a=c.P;20(a&&a!=R){b.Y(a);a=a.P}F b},25:9(a,e,c,b){e=e||1;E d=0;I(;a;a=a[c])6(a.1t==1&&++d==e)1J;F a},4A:9(n,a){E r=[];I(;n;n=n.2a){6(n.1t==1&&(!a||n!=a))r.Y(n)}F r}});C.1c={1r:9(f,d,c,b){6(C.N.12&&f.3t!=Q)f=19;6(!c.22)c.22=7.22++;6(b!=Q){E e=c;c=9(){F e.T(7,1a)};c.V=b;c.22=e.22}6(!f.$1i)f.$1i={};6(!f.$1y)f.$1y=9(){E a;6(1f C=="Q"||C.1c.4n)F a;a=C.1c.1y.T(f,1a);F a};E g=f.$1i[d];6(!g){g=f.$1i[d]={};6(f.4m)f.4m(d,f.$1y,M);G f.7M("3r"+d,f.$1y)}g[c.22]=c;7.1D[d]=O},22:1,1D:{},23:9(c,b,a){E d=c.$1i,2c,45;6(d){6(b&&b.L){a=b.4l;b=b.L}6(!b){I(b 17 d)7.23(c,b)}G 6(d[b]){6(a)4k d[b][a.22];G I(a 17 c.$1i[b])4k d[b][a];I(2c 17 d[b])1J;6(!2c){6(c.4j)c.4j(b,c.$1y,M);G c.7L("3r"+b,c.$1y);2c=K;4k d[b]}}I(2c 17 d)1J;6(!2c)c.$1y=c.$1i=K}},1z:9(c,b,d){b=C.2V(b||[]);6(!d){6(7.1D[c])C("*").1r([19,R]).1z(c,b)}G{E a,2c,15=C.1g(d[c]||K);b.42(7.4i({L:c,1S:d}));6(C.1g(d.$1y))a=d.$1y.T(d,b);6(!15&&d["3r"+c]&&d["3r"+c].T(d,b)===M)a=M;6(15&&a!==M&&!(C.W(d,\'a\')&&c=="4h")){7.4n=O;d[c]()}7.4n=M}},1y:9(b){E a;b=C.1c.4i(b||19.1c||{});E c=7.$1i&&7.$1i[b.L],2e=1K.3v.3S.2S(1a,1);2e.42(b);I(E j 17 c){2e[0].4l=c[j];2e[0].V=c[j].V;6(c[j].T(7,2e)===M){b.2u();b.2X();a=M}}6(C.N.12)b.1S=b.2u=b.2X=b.4l=b.V=K;F a},4i:9(c){E a=c;c=C.14({},a);c.2u=9(){6(a.2u)a.2u();a.7I=M};c.2X=9(){6(a.2X)a.2X();a.7H=O};6(!c.1S&&c.5r)c.1S=c.5r;6(C.N.1H&&c.1S.1t==3)c.1S=a.1S.P;6(!c.4g&&c.4F)c.4g=c.4F==c.1S?c.7C:c.4F;6(c.5p==K&&c.66!=K){E e=R.3D,b=R.4z;c.5p=c.66+(e&&e.5o||b.5o||0);c.7z=c.7v+(e&&e.5m||b.5m||0)}6(!c.3Q&&(c.5k||c.5j))c.3Q=c.5k||c.5j;6(!c.5i&&c.5g)c.5i=c.5g;6(!c.3Q&&c.1X)c.3Q=(c.1X&1?1:(c.1X&2?3:(c.1X&4?2:0)));F c}};C.15.14({3l:9(c,a,b){F c=="5f"?7.5e(c,a,b):7.J(9(){C.1c.1r(7,c,b||a,b&&a)})},5e:9(d,b,c){F 7.J(9(){C.1c.1r(7,d,9(a){C(7).49(a);F(c||b).T(7,1a)},c&&b)})},49:9(a,b){F 7.J(9(){C.1c.23(7,a,b)})},1z:9(a,b){F 7.J(9(){C.1c.1z(a,b,7)})},1W:9(){E a=1a;F 7.4h(9(e){7.3T=0==7.3T?1:0;e.2u();F a[7.3T].T(7,[e])||M})},7p:9(f,g){9 3U(e){E p=e.4g;20(p&&p!=7)2g{p=p.P}2h(e){p=7};6(p==7)F M;F(e.L=="3W"?f:g).T(7,[e])}F 7.3W(3U).5b(3U)},1L:9(f){5a();6(C.36)f.T(R,[C]);G C.2C.Y(9(){F f.T(7,[C])});F 7}});C.14({36:M,2C:[],1L:9(){6(!C.36){C.36=O;6(C.2C){C.J(C.2C,9(){7.T(R)});C.2C=K}6(C.N.3J||C.N.2t)R.4j("59",C.1L,M);6(!19.7m.H)C(19).2f(9(){C("#4b").23()})}}});C.J(("7l,7k,2f,7j,7i,5f,4h,7g,"+"7f,7d,7c,3W,5b,7b,2b,"+"4u,7a,79,78,3f").2M(","),9(i,o){C.15[o]=9(f){F f?7.3l(o,f):7.1z(o)}});E w=M;9 5a(){6(w)F;w=O;6(C.N.3J||C.N.2t)R.4m("59",C.1L,M);G 6(C.N.12){R.75("<73"+"72 2j=4b 70=O "+"32=//:><\\/33>");E a=R.37("4b");6(a)a.6Z=9(){6(R.3d!="1x")F;C.1L()};a=K}G 6(C.N.1H)C.48=3t(9(){6(R.3d=="6Y"||R.3d=="1x"){47(C.48);C.48=K;C.1L()}},10);C.1c.1r(19,"2f",C.1L)}C.15.14({6X:9(c,b,a){7.2f(c,b,a,1)},2f:9(g,e,c,d){6(C.1g(g))F 7.3l("2f",g);c=c||9(){};E f="46";6(e)6(C.1g(e)){c=e;e=K}G{e=C.2O(e);f="55"}E h=7;C.31({1G:g,L:f,V:e,2F:d,1x:9(a,b){6(b=="1U"||!d&&b=="54")h.5W(a.43);4x(9(){h.J(c,[a.43,b,a])},13)}});F 7},6W:9(){F C.2O(7)},6V:9(){}});C.J("53,52,51,50,4Z,5h".2M(","),9(i,o){C.15[o]=9(f){F 7.3l(o,f)}});C.14({21:9(e,c,a,d,b){6(C.1g(c)){a=c;c=K}F C.31({L:"46",1G:e,V:c,1U:a,3G:d,2F:b})},6U:9(d,b,a,c){F C.21(d,b,a,c,1)},6T:9(b,a){F C.21(b,K,a,"33")},77:9(c,b,a){F C.21(c,b,a,"56")},6S:9(d,b,a,c){6(C.1g(b)){a=b;b={}}F C.31({L:"55",1G:d,V:b,1U:a,3G:c})},6R:9(a){C.3u.1Q=a},6Q:9(a){C.14(C.3u,a)},3u:{1D:O,L:"46",1Q:0,4Y:"6P/x-6O-38-6N",4X:O,2w:O,V:K},3h:{},31:9(s){s=C.14(O,s,C.14(O,{},C.3u,s));6(s.V){6(s.4X&&1f s.V!="1E")s.V=C.2O(s.V);6(s.L.2D()=="21"){s.1G+=(s.1G.U("?")>-1?"&":"?")+s.V;s.V=K}}6(s.1D&&!C.40++)C.1c.1z("53");E f=M;E h=19.4W?1s 4W("6M.6K"):1s 58();h.6J(s.L,s.1G,s.2w);6(s.V)h.4c("7r-7s",s.4Y);6(s.2F)h.4c("6G-3Z-6E",C.3h[s.1G]||"7w, 6C 7y 6B 4J:4J:4J 6z");h.4c("X-7D-7E","58");6(s.4U)s.4U(h);6(s.1D)C.1c.1z("5h",[h,s]);E g=9(d){6(!f&&h&&(h.3d==4||d=="1Q")){f=O;6(i){47(i);i=K}E c=d=="1Q"&&"1Q"||!C.5n(h)&&"3f"||s.2F&&C.5s(h,s.1G)&&"54"||"1U";6(c=="1U"){2g{E a=C.5q(h,s.3G)}2h(e){c="4I"}}6(c=="1U"){E b;2g{b=h.4f("4S-3Z")}2h(e){}6(s.2F&&b)C.3h[s.1G]=b;6(s.1U)s.1U(a,c);6(s.1D)C.1c.1z("4Z",[h,s])}G C.3X(s,h,c);6(s.1D)C.1c.1z("51",[h,s]);6(s.1D&&!--C.40)C.1c.1z("52");6(s.1x)s.1x(h,c);6(s.2w)h=K}};6(s.2w){E i=3t(g,13);6(s.1Q>0)4x(9(){6(h){h.6w();6(!f)g("1Q")}},s.1Q)}2g{h.6v(s.V)}2h(e){C.3X(s,h,K,e)}6(!s.2w)g();F h},3X:9(s,a,b,e){6(s.3f)s.3f(a,b,e);6(s.1D)C.1c.1z("50",[a,s,e])},40:0,5n:9(r){2g{F!r.26&&6t.6r=="4v:"||(r.26>=4R&&r.26<6q)||r.26==5z||C.N.1H&&r.26==Q}2h(e){}F M},5s:9(a,c){2g{E b=a.4f("4S-3Z");F a.26==5z||b==C.3h[c]||C.N.1H&&a.26==Q}2h(e){}F M},5q:9(r,a){E b=r.4f("6o-L");E c=a=="5F"||!a&&b&&b.U("5F")>=0;V=c?r.7W:r.43;6(c&&V.3D.4y=="4I")7X"4I";6(a=="33")C.4E(V);6(a=="56")V=2T("("+V+")");F V},2O:9(a){E s=[];6(a.1b==1K||a.3w)C.J(a,9(){s.Y(2y(7.6l)+"="+2y(7.2A))});G I(E j 17 a)6(a[j]&&a[j].1b==1K)C.J(a[j],9(){s.Y(2y(j)+"="+2y(7))});G s.Y(2y(j)+"="+2y(a[j]));F s.5w("&")}});C.15.14({1o:9(b,a){F b?7.1B({1u:"1o",24:"1o",1e:"1o"},b,a):7.1l(":1C").J(9(){7.S.11=7.2r?7.2r:"";6(C.1h(7,"11")=="1T")7.S.11="2m"}).3L()},1p:9(b,a){F b?7.1B({1u:"1p",24:"1p",1e:"1p"},b,a):7.1l(":4N").J(9(){7.2r=7.2r||C.1h(7,"11");6(7.2r=="1T")7.2r="2m";7.S.11="1T"}).3L()},5O:C.15.1W,1W:9(a,b){F C.1g(a)&&C.1g(b)?7.5O(a,b):a?7.1B({1u:"1W",24:"1W",1e:"1W"},a,b):7.J(9(){C(7)[C(7).3y(":1C")?"1o":"1p"]()})},6i:9(b,a){F 7.1B({1u:"1o"},b,a)},6h:9(b,a){F 7.1B({1u:"1p"},b,a)},6g:9(b,a){F 7.1B({1u:"1W"},b,a)},6f:9(b,a){F 7.1B({1e:"1o"},b,a)},89:9(b,a){F 7.1B({1e:"1p"},b,a)},6e:9(c,a,b){F 7.1B({1e:a},c,b)},1B:9(d,h,f,g){F 7.1n(9(){E c=C(7).3y(":1C"),1Z=C.5V(h,f,g),5U=7;I(E p 17 d){6(d[p]=="1p"&&c||d[p]=="1o"&&!c)F C.1g(1Z.1x)&&1Z.1x.T(7);6(p=="1u"||p=="24"){1Z.11=C.1h(7,"11");1Z.2z=7.S.2z}}6(1Z.2z!=K)7.S.2z="1C";7.2v=C.14({},d);C.J(d,9(a,b){E e=1s C.2Y(5U,1Z,a);6(b.1b==3x)e.3R(e.1Y()||0,b);G e[b=="1W"?c?"1o":"1p":b](d)});F O})},1n:9(a,b){6(!b){b=a;a="2Y"}F 7.J(9(){6(!7.1n)7.1n={};6(!7.1n[a])7.1n[a]=[];7.1n[a].Y(b);6(7.1n[a].H==1)b.T(7)})}});C.14({5V:9(b,a,c){E d=b&&b.1b==8G?b:{1x:c||!c&&a||C.1g(b)&&b,1N:b,35:c&&a||a&&a.1b!=8F&&a};d.1N=(d.1N&&d.1N.1b==3x?d.1N:{8D:8C,8B:4R}[d.1N])||8A;d.2U=d.1x;d.1x=9(){C.68(7,"2Y");6(C.1g(d.2U))d.2U.T(7)};F d},35:{62:9(p,n,b,a){F b+a*p},4H:9(p,n,b,a){F((-67.8z(p*67.8y)/2)+0.5)*a+b}},1n:{},68:9(b,a){a=a||"2Y";6(b.1n&&b.1n[a]){b.1n[a].4s();E f=b.1n[a][0];6(f)f.T(b)}},3N:[],2Y:9(f,e,g){E z=7;E y=f.S;z.a=9(){6(e.3q)e.3q.T(f,[z.2x]);6(g=="1e")C.1j(y,"1e",z.2x);G{y[g]=5K(z.2x)+"4t";6(g=="1u"||g=="24")y.11="2m"}};z.65=9(){F 3m(C.1h(f,g))};z.1Y=9(){E r=3m(C.34(f,g));F r&&r>-8v?r:z.65()};z.3R=9(c,b){z.4M=(1s 64()).63();z.2x=c;z.a();C.3N.Y(9(){F z.3q(c,b)});6(C.3N.H==1){E d=3t(9(){E a=C.3N;I(E i=0;i<a.H;i++)6(!a[i]())a.8t(i--,1);6(!a.H)47(d)},13)}};z.1o=9(){6(!f.2i)f.2i={};f.2i[g]=C.1j(f.S,g);e.1o=O;z.3R(0,7.1Y());6(g!="1e")y[g]="8r";C(f).1o()};z.1p=9(){6(!f.2i)f.2i={};f.2i[g]=C.1j(f.S,g);e.1p=O;z.3R(7.1Y(),0)};z.3q=9(a,c){E t=(1s 64()).63();6(t>e.1N+z.4M){z.2x=c;z.a();6(f.2v)f.2v[g]=O;E b=O;I(E i 17 f.2v)6(f.2v[i]!==O)b=M;6(b){6(e.11!=K){y.2z=e.2z;y.11=e.11;6(C.1h(f,"11")=="1T")y.11="2m"}6(e.1p)y.11="1T";6(e.1p||e.1o)I(E p 17 f.2v)C.1j(y,p,f.2i[p])}6(b&&C.1g(e.1x))e.1x.T(f);F M}G{E n=t-7.4M;E p=n/e.1N;z.2x=C.35[e.35||(C.35.4H?"4H":"62")](p,n,a,(c-a),e.1N);z.a()}F O}}})})();',62,541,'||||||if|this||function|||||||||||||||||||||||||||||||var|return|else|length|for|each|null|type|false|browser|true|parentNode|undefined|document|style|apply|indexOf|data|nodeName||push|||display|msie||extend|fn|jQuery|in|className|window|arguments|constructor|event|test|opacity|typeof|isFunction|css|events|attr|firstChild|filter|div|queue|show|hide|mergeNum|add|new|nodeType|height|replace|tbody|complete|handle|trigger|table|animate|hidden|global|string|find|url|safari|toUpperCase|break|Array|ready|al|duration|pushStack|tb|timeout|stack|target|none|success|swap|toggle|button|cur|opt|while|get|guid|remove|width|nth|status|checked|selected|merge|nextSibling|select|ret|exec|args|load|try|catch|orig|id|match|_|block||rl|insertBefore|done|oldblock|trim|opera|preventDefault|curAnim|async|now|encodeURIComponent|overflow|value|grep|readyList|toLowerCase|color|ifModified|val|first|ownerDocument|domManip|substr|defaultView|split|has|param|last|text|multiFilter|call|eval|old|makeArray|innerHTML|stopPropagation|fx|childNodes|disabled|ajax|src|script|curCSS|easing|isReady|getElementById|form|input|float|getComputedStyle|clean|readyState|removeChild|error|static|lastModified|checkbox|selectedIndex|position|bind|parseFloat|String|oWidth|oHeight|step|on|toString|setInterval|ajaxSettings|prototype|jquery|Number|is|child|ol|cloneNode|RegExp|documentElement|isXMLDoc|getAttribute|dataType|append|styleFloat|mozilla|empty|end|map|timers|tr|el|which|custom|slice|lastToggle|handleHover|visibility|mouseover|handleError|lastChild|Modified|active|currentStyle|unshift|responseText|getPropertyValue|index|GET|clearInterval|safariTimer|unbind|init|__ie_init|setRequestHeader|unique|radio|getResponseHeader|relatedTarget|click|fix|removeEventListener|delete|handler|addEventListener|triggered|nodeIndex|appendChild|props|classFilter|shift|px|submit|file|expr|setTimeout|tagName|body|sibling|previousSibling|parents|deep|globalEval|fromElement|cssFloat|swing|parsererror|00|inArray|getElementsByTagName|startTime|visible|num|object|prop|200|Last|colgroup|beforeSend|fieldset|ActiveXObject|processData|contentType|ajaxSuccess|ajaxError|ajaxComplete|ajaxStop|ajaxStart|notmodified|POST|json|appendTo|XMLHttpRequest|DOMContentLoaded|bindReady|mouseout|prevObject|removeAttr|one|unload|ctrlKey|ajaxSend|metaKey|keyCode|charCode|not|scrollTop|httpSuccess|scrollLeft|pageX|httpData|srcElement|httpNotModified|after|before|prepend|join|parse|zoom|304|reset|image|password|odd|even|xml|quickClass|quickID|quickChild|setArray|parseInt|contains|gt|execScript|_toggle|lt|eq|href|nodeValue|alpha|self|speed|html|continue|parent|textContent|createTextNode|webkit|linear|getTime|Date|max|clientX|Math|dequeue|fl|createElement|version|100|NaN|fadeTo|fadeIn|slideToggle|slideUp|slideDown|setAttribute|getAttributeNode|name|method|action|content|cssText|300|protocol|FORM|location|options|send|abort|col|th|GMT|td|1970|01|cap|Since|colg|If|tfoot|thead|open|XMLHTTP|leg|Microsoft|urlencoded|www|application|ajaxSetup|ajaxTimeout|post|getScript|getIfModified|evalScripts|serialize|loadIfModified|loaded|onreadystatechange|defer|clientWidth|ipt|scr|clientHeight|write|relative|getJSON|keyup|keypress|keydown|change|mousemove|mouseup|left|mousedown|dblclick|right|scroll|resize|focus|blur|frames|absolute|clone|hover|offsetWidth|Content|Type|offsetHeight|Width|clientY|Thu|border|Jan|pageY|padding|Left|toElement|Requested|With|Right|Bottom|cancelBubble|returnValue|Top|size|detachEvent|attachEvent|substring|line|textarea|weight|enabled|font|innerText|only|uFFFF|responseXML|throw|u0128|417|toggleClass|removeClass|wrap|addClass|removeAttribute|insertAfter|prependTo|children|siblings|fadeOut|noConflict|prev|next|Boolean|maxLength|maxlength|readOnly|readonly|class|htmlFor|CSS1Compat|compatMode|boxModel|compatible|ie|ra|it|1px|rv|splice|userAgent|10000|navigator|concat|PI|cos|400|fast|600|slow|reverse|Function|Object|array|ig'.split('|'),0,{})) \ No newline at end of file
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/static/layout.css @@ -0,0 +1,42 @@ +html, body { margin: 0; padding: 0; } +body { background: #ddd; color: #333; font: normal 90%/1.3 Arial,Helvetica,sans-serif; } +:link, :visited { color: #c10000; text-decoration: none; } +:link:hover, :visited:hover { text-decoration: underline; } +:link img, :visited img { border: none; } +h1 { color: #666; + font: normal xx-large/1.5 Georgia,serif; margin: 0 0 .5em; +} +blockquote { font-style: italic; } +form table { margin-bottom: 1em; } +form table tbody th { font-weight: normal; padding-top: .3em; text-align: right; + vertical-align: top; +} +form label { color: #666; font-size: 90%; } + +#wrap { background: #fff; width: 600px; margin: 30px auto; } +#content { border-left: 10px solid #b00; min-height: 240px; padding: 10px; } +#header img { float: right; margin-top: -21px; margin-right: -37px; } + +#footer { background: #4A4D4D; border-top: 2px solid #333; font-size: x-small; + padding: 3px; text-align: center; +} +#footer hr { display: none; } +.legalese { color: #999; margin: 0; } + +ol.links li .info { font-size: 85%; } +ol.links li .info :link, ol.links li .info :visited { color: #666; } + +ul.comments { list-style: none; margin: 1em 0; padding: 0 0 0 1em; } +ul.comments li { color: #999; margin: 0 0 1em; } +ul.comments blockquote { color: #333; font-style: normal; margin: 0; + padding: 0; +} +.action:link, .action:visited { background: #f3f3f3; border: 1px outset #ddd; + color: #666; font-size: 90%; padding: 0 .3em; +} +.action:link:hover, .action:visited:hover { background: #e8e8e8; + border-color: #aaa; color: #000; text-decoration: none; +} + +form p.hint { color: #666; font-size: 90%; font-style: italic; margin: 0; } +form .error { color: #b00; }
new file mode 100644 index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..60267e548999abf7ddb818be82c0e9397ef51b7b GIT binary patch literal 3266 zc$@*k3_bHlNk%w1Vd(%{0K@<Q0002*@9)ga%xi0FGcz--t*zbN-ISD+5D*YjQd0l_ z|Ad5uy}i9cLPFNo)*&GwW@cukrKNj&dk6>!#l^+7wY5r0N;*0^nVFg9=H^;jTKD(& zjEszPb8{IP87V0#A^8LV2LS&7EC2ui0O<f*000I5pdpTAX`X1Ru59bRa4f@4Z3}pA z*K_axz@TtQ9Nlurq%z4XI-k&}G*(+$uh!;t%k6qu4lI&VDVQq^ig@Yxy5DeOSWJmX zI{3Vv!&o!hae;z^C39F55)c}T8W00|c@Q0aema1Jn3+?CSP=;p772|55(^v$sHhwZ z5(5~OIBJ)gw6#B+R|gHdq8$qd5fKFy#Ki>>2fwfaX#%shB?1+~3_c7649`#wWfXCh z6&Du`2@nz(!_7++*BBAY41L-`T0jgC4*>f56CE!O7!r8c2jGE*2u~1Qn6Pic1qs!< zz-llq!2k{sn%OX6gI&gD!zj#yAz*<90tg8IBKUA{h5!u&V*>EDB4A3H`#cN~5Cjnc zoCFCR5W?VR0-hvbMG$9^12$(ZYEUS!OhAMRB70!?36$W4kti7C>k6Pk3I-Z(l5Ge9 zsf+^`*2a7=)JFuh1d`%dK#Bu8N=Oo1umE9!urkONu#!;sgaZm{U{N@L6)cJfW98~g z@Pfj?m<S$-eqbTRfwdkO%JhIRMgy`FKFe(27Dfcs`aT$`F#zZU1RjD<7(RS0*vC~P zV5k5!Y66@N6D0T`6Tu7W`yMQ)y}|-ep-3ZmmnngUZw&=Rl+2vdEJ?MSMRe3mU*Cll zxco@?*g~w#4+qR5K};UF(Az#=#Q*{SP79n+0apb1^MY(SMO4B9xaos{doC;(Q-cQN zKtX+)AwW|K`K_@^0a>X~0ek>VP~Hh05Y(6o6yWxNWoN-e5QirCNDv4AU`CK`VX+5* z2i&RfTucaYq9lni5n!PTDPmE<0F*W7AdCp4mqKp{+~_1hV!9L)kS2K8(3LC<xDS#j z2%uyGFJK^(S)6d-!2=%<An2e3+}S}Xr%=Nq30Y#HfnhD0U?*s7o{)_T0^o+{4-{5f z!jlNvcH|14#bl-maS`OrGN;+NC<|4+)By(B%<~#S6Qnsrjr+9b1B7Mar=+heFfi&0 z5zO_f0R_b5%reRTxU34X4gdlF2h5$YfJ~<bky(L2fEw%y&3g6Eu`7rR0FhC}R0S** z#1yFyu}YwU1S~!y0E`+uVeCHr0^#Pk-Ngi_3Tn|wZH9Td5R(Cou0xOq^Y#$y0ts{w zLB#_XH0=v=>VrWLArtIkOcJ($9RLce(D8{Y;0ke71U*o(OI3A{F|D&?8q>&6+^SEL zZ~f>(s`@nC0@5v?6%)HGs5J93LES7vtPoh>%B3Ul8Y&bz3K6Y7o@y(z3*)i;0<ZzI zfE&0~X=*hMu{NMJ&jI8efRYvH?f2h+2M+j%`y?Rv;e!9G4+Dt%?SQ+D>+L21k?ZX> z01Z?gfLagaoxw7=X^_GH-x<gdu@!@D?SKNT3lLKStj8|<?6lWD``7?TVEgX28$gf% zyw6S{1;NK2G6BWMt~krdD<Inkv2PHf7zS*hmhQ6$aCKD$kTYvG#>Z4Z^W=x$Yx%ne z@H_cJm(@H6`yk)E?>RM}nf%e0{w`Fbvo0G4w9{S;uttvkh$MDMTha#5M?N6|fB_7w z-3Hdzyn&4`d94Bf{*tGX=B;dl<dY2s0!W3bP+%O7$`|W4kRb>}uz3kw-3w<&!Q@>o z0?pG?(#ofn33w1e5tI!Mvp2x13}7(V$`<Pgw+{%;FnAOSz!PPM8{jo>TjxvO*gzP( z%=j>Q5P}^8GgATo68@tq;kdxgYQ;Jo@?(Kh++Bv^7`rNx4~zP^VDYwCzRVz?d0~uT zW}GNH0Y;^6NL)a^48Xd}S%8kVo8UeW;D7`?AUK%JB;}H4$19?#d3+?F9|5_KM~-nH z#3Lf6sv;Qg(F!T7b0GF0IlBztDnmFyfE0_@$>f!+j9QFe`DEh(QtHE$=*x^13+aN> zDM15Fxz-{XsetzBk(RNeUwsG}yc!noa@^!yD4WMQV*(%nLEMK7nF$6~ii0qQ;Tko8 zwN7lZ?s4?l5c#|ryn!t<cy8n(?{Fi~+Cd1N43rHu*@>lPRDdE#Jcm3+z`B(IV3+o6 z9j-(H0Y{Yo0vHU)K>7H|J8>-(pu%jPB=sT2;vF=CY7wa_QM8N&h{JB?SdN#d$)1eL z&UN*p<4OVW&)~HWj>XGhL2IW?0BFFR2t^-mWXe#-z{>(REyn_0`3%-U=#M_t=p-Sj zzI_yxcPj%c22+|mCIX-X=6osx34+ns8B$V}puiEwVF3*cAf7uF0qcbJ#jxU&A7mw| z9Opzn`0WssLy9W{%c_sMvhJ>W(T_MPVvYkmWgf94$O{D<yu<1PsZX?}I_F1_s1^^8 zOQqc}C5yhMT2=v=RgWKRnpF+>;wrL00qgL1Q3j^!0Krr3K2AErrq+{p<<iG2ix;vB zkS}fj0ATL!ASBoF0rj+I%q$joTM@r*^|LBqUA&fCTYv?RvWugmX`j2h**4E>)@>d! z$%nM=l5csay=*kZKwfhAl`$C+36EH(+y|mhaV=f!K1LYF?*f3nwgc8}cXuEF<hOXH z^>6U{%fRRYAi%g4?-Jm0)4q}d7+49ca4WhG0G4lTAKo2%20%v(W0-ag&T;YFB;?&W zR%HC$hYQh{K>*awkd5kIiw_Lr;JUzk=R_YKi%dK0#`eDi(QfUUjNAt9Bf`9c&;&MY zJE1MPd?kCbe6FxpyEOvAF|G#-Vt3dHpqO_={$@<rT)y?{L(H^OGgXrZDm2^9!mcg< zo|_psFga)Gp;<KtM)LeI(y}gX6gDT9XJ=vngnEW1Mzrh<4S+W{FKv#low!ojyh)?4 zeYe$~6<BO%U!X%4ou+8)nD~G{*KT=V#_j9kh&xrU(4s%yozP-EyX8@PcWHC=56v3D z6!DfEo&9XENiaK5Y1?)gU{A6YkU*kN(T&*+-sO0VKBQgyvg|o)@Unf%2?uZq9LVZl zfve{k)KV=FFgEKLKp_n#58jxY;*l@RJ9v9w(cAVg?>5lBa~IFt#`OyAriX+yde12n z6$0%Tpd$?2mI3O@P4cW~swb8#oU$v~1ZX|X?Cj0}3A}*9cPAu|%Nt6w%MVunPtGD4 z`=E9Oz$IxG(Eu3L?0qqr05BuiJ{yx7SQLw3g8_hX1!Q0W@UZ})0j*>Px-~|pRssep za*=5g!FDgAl0J%=aekB;eHqA5nIKaD4kQ3Z4Jf-Oy@C}6M4$uho`BaG*kO9tTP6?q zs{|6j0VM1Ko*NtY3Vz<ce3H)qR=)tw_f4V)ToZi`usT^dWAlAXmwZ0!3T<!@jt1Q8 zl~4#C%yF5WM!;w2v)dO=9smK#^g;2g!`)gwFvJ;_o-7)OHu*9*`SLYcCKkwnO$p-^ zrw3u?R!Igh1b|f2Lf>`{09IFCA3Ny-B>`K1;`LYK4@erC{*BcY2ap5*7T`+q<OGle zA=+m<ezF#~_frQj01dz%6JP-@M@t(}0VZ()@ZkXvuzIPr5AWAj7R55JkOL^BNalBb zbR;Ao7(5%`2@Q8lYbQG!APp3Nd~}onWZ-O51OYNaM-;#T9ng76Kmj<_B~k!MAp?Nr z!-Hjj4I>y<R9J-!lmrj8MxxgOkQ80jXN6!GhGO_=NDu)96naZYN80p-W7vjnNQIn$ z0ltSXbE9&QG!{<yhI+V%FC;w`fB_}}PgJpmdj?5*@&Pr-hl;p}yORM0umBLy3rd)V zj97`5*oGPKhj&nf89|7Jc!{7GiarH}%JCN~XosShimLcT0T2fN?4S`_u!^!ci{zs{ zJ5X>>K#RJ#i{a%4y!eZ{Xp1}G1;AL0qNr<EfQxr$jLf)*8{kYj5CzRRjd~aW@L**{ zK#kfchBt^tDYuN<7><S&D*~__b!c$nn2y+FjpV2t%;AaTrjGI$M<i!L5=AsUFpv5; zL*$qPg<*F<u#W<XKK+OTYlM$7FpvtFJOwEL4e*Z$83PMBk-Gzp>oA9dh>8@skqp+4 za+m-H1dl$*k^5+oNw5p|0+3Hfk|fEF%OC+Mn2{{Vk{tOturPOO#3g+&1T@K#t>^~< zhe9FAlRZh28(@=GPyvF0LbLLYL|K$RiIil(0a(RXzU5gz=O~d-*^>oOfV=ROp%89a zsg+yFk_=!ij4%okuxVpSmSxG20&oDUpctkA4CsY`Z%L8{a1CbAml`3GCJ+JuJ6^Mt AyZ`_I
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/templates/_comment.html @@ -0,0 +1,5 @@ +<?python from genshi import HTML ?> +<li id="comment$num"> + <strong>${comment.username}</strong> at ${comment.time.strftime('%x %X')} + <blockquote>${HTML(comment.content)}</blockquote> +</li>
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/templates/_form.html @@ -0,0 +1,24 @@ +<form xmlns="http://www.w3.org/1999/xhtml" + xmlns:py="http://genshi.edgewall.org/" + class="comment" action="${url('/comment/%s/' % link.id)}" method="post"> + <table summary=""><tbody><tr> + <th><label for="username">Your name:</label></th> + <td> + <input type="text" id="username" name="username" /> + <span py:if="'username' in errors" class="error">${errors.username}</span> + </td> + </tr><tr> + <th><label for="comment">Comment:</label></th> + <td> + <textarea id="comment" name="content" rows="6" cols="50"></textarea> + <span py:if="'content' in errors" class="error"><br />${errors.content}</span> + <p class="hint">You can use HTML tags here for formatting.</p> + </td> + </tr><tr> + <td></td> + <td> + <input type="submit" value="Submit" /> + <input type="submit" name="cancel" value="Cancel" /> + </td> + </tr></tbody></table> +</form>
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/templates/comment.html @@ -0,0 +1,18 @@ +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml" + xmlns:xi="http://www.w3.org/2001/XInclude" + xmlns:py="http://genshi.edgewall.org/"> + <xi:include href="layout.html" /> + <head> + <title>Comment on “${link.title}”</title> + </head> + <body class="comment"> + <h1>Comment on “${link.title}”</h1> + <p py:if="comment"> + In reply to <strong>${comment.username}</strong> + at ${comment.time.strftime('%x %X')}: + <blockquote>${comment.content}</blockquote> + </p> + <xi:include href="_form.html" /> + </body> +</html>
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/templates/index.html @@ -0,0 +1,28 @@ +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml" + xmlns:xi="http://www.w3.org/2001/XInclude" + xmlns:py="http://genshi.edgewall.org/"> + <xi:include href="layout.html" /> + <head> + <title>News</title> + <link rel="alternate" type="application/atom+xml" title="Geddit" + href="${url('/feed/')}" /> + </head> + <body class="index"> + <h1>News</h1> + + <ol py:if="links" class="links"> + <li py:for="link in links"> + <a href="${link.url}">${link.title}</a> + posted by ${link.username} at ${link.time.strftime('%x %X')} + <div class="info"> + <a href="${url('/info/%s/' % link.id)}"> + ${len(link.comments)} comments + </a> + </div> + </li> + </ol> + + <p><a class="action" href="${url('/submit/')}">Submit new link</a></p> + </body> +</html>
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/templates/index.xml @@ -0,0 +1,23 @@ +<?xml version="1.0" encoding="utf-8"?> +<feed xmlns="http://www.w3.org/2005/Atom" + xmlns:py="http://genshi.edgewall.org/"> + + <title>Geddit News</title> + <id href="${url('/')}"/> + <link rel="alternate" href="${url('/')}" type="text/html"/> + <link rel="self" href="${url('/feed/')}" type="application/atom+xml"/> + <updated>${links[0].time.isoformat()}</updated> + + <entry py:for="link in reversed(links)"> + <title>${link.url}</title> + <link rel="alternate" href="${link.url}" type="text/html"/> + <link rel="via" href="${url('/info/%s/' % link.id)}" type="text/html"/> + <id>${url('/info/%s/' % link.id)}</id> + <author> + <name>${link.username}</name> + </author> + <updated>${link.time.isoformat()}</updated> + <summary>${link.title}</summary> + </entry> + +</feed>
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/templates/info.html @@ -0,0 +1,62 @@ +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml" + xmlns:xi="http://www.w3.org/2001/XInclude" + xmlns:py="http://genshi.edgewall.org/"> + <xi:include href="layout.html" /> + <head> + <title>${link.title}</title> + <link rel="alternate" type="application/atom+xml" title="Geddit: ${link.title}" + href="${url('/feed/%s/' % link.id)}" /> + <script type="text/javascript"> + function loadCommentForm(a) { + $.get("${url('/comment/%s/' % link.id)}", {}, function(html) { + var form = a.hide().parent().after(html).next(); + function closeForm() { + form.slideUp("fast", function() { a.fadeIn(); form.remove() }); + return false; + } + function initForm() { + form.find("input[@name='cancel']").click(closeForm); + form.submit(function() { + var data = form.find("input[@type='text'], textarea").serialize(); + $.post("${url('/comment/%s/' % link.id)}", data, function(html) { + var elem = $(html).get(0); + if (/form/i.test(elem.tagName)) { + form.after(elem).remove(); + form = $(elem); + initForm(); + } else { + if ($("ul.comments").length == 0) { + a.parent().before('<ul class="comments"></ul>'); + } + $("ul.comments").append($(elem)); + closeForm(); + } + }); + return false; + }); + } + initForm(); + }); + } + $(document).ready(function() { + $("a.action").click(function() { + loadCommentForm($(this)); + return false; + }); + }); + </script> + </head> + <body class="info"> + <h1>${link.title}</h1> + <a href="${link.url}">${link.url}</a><br /> + posted by ${link.username} at ${link.time.strftime('%x %X')}<br /> + + <ul py:if="link.comments" class="comments"> + <xi:include href="_comment.html" + py:for="num, comment in enumerate(link.comments)" /> + </ul> + + <p><a class="action" href="${url('/comment/%s/' % link.id)}">Add comment</a></p> + </body> +</html>
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/templates/info.xml @@ -0,0 +1,28 @@ +<?xml version="1.0" encoding="utf-8"?> +<feed xmlns="http://www.w3.org/2005/Atom" + xmlns:py="http://genshi.edgewall.org/"> + + <title>Geddit: ${link.title}</title> + <id href="${url('/info/%s/' % link.id)}"/> + <link rel="alternate" href="${url('/info/%s/' % link.id)}" type="text/html"/> + <link rel="self" href="${url('/feed/%s/' % link.id)}" type="application/atom+xml"/> + <updated py:with="time=link.comments and link.comments[-1].time or link.time"> + ${time.isoformat()} + </updated> + + <?python from genshi import HTML ?> + <entry py:for="idx, comment in enumerate(reversed(link.comments))"> + <title>Comment ${len(link.comments) - idx} on “${link.title}”</title> + <link rel="alternate" href="${url('/info/%s/' % link.id)}#comment${idx}" + type="text/html"/> + <id>${url('/info/%s/' % link.id)}#comment${idx}</id> + <author> + <name>${comment.username}</name> + </author> + <updated>${comment.time.isoformat()}</updated> + <content type="xhtml"><div xmlns="http://www.w3.org/1999/xhtml"> + ${HTML(comment.content)} + </div></content> + </entry> + +</feed>
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/templates/layout.html @@ -0,0 +1,31 @@ +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml" + xmlns:py="http://genshi.edgewall.org/" py:strip=""> + + <py:match path="head" once="true"> + <head py:attrs="select('@*')"> + <title py:with="title = list(select('title/text()'))"> + Geddit<py:if test="title">: ${title}</py:if> + </title> + <link rel="stylesheet" href="${url('/media/layout.css')}" type="text/css" /> + <script type="text/javascript" src="${url('/media/jquery.js')}"></script> + ${select('*[local-name()!="title"]')} + </head> + </py:match> + + <py:match path="body" once="true"> + <body py:attrs="select('@*')"><div id="wrap"> + <div id="header"> + <a href="/"><img src="${url('/media/logo.gif')}" width="201" height="79" alt="geddit?" /></a> + </div> + <div id="content"> + ${select('*|text()')} + </div> + <div id="footer"> + <hr /> + <p class="legalese">© 2007 Edgewall Software</p> + </div> + </div></body> + </py:match> + +</html>
new file mode 100644 --- /dev/null +++ b/examples/tutorial/geddit/templates/submit.html @@ -0,0 +1,41 @@ +<!DOCTYPE html> +<html xmlns="http://www.w3.org/1999/xhtml" + xmlns:xi="http://www.w3.org/2001/XInclude" + xmlns:py="http://genshi.edgewall.org/"> + <xi:include href="layout.html" /> + <head> + <title>Submit new link</title> + </head> + <body class="submit"> + <h1>Submit new link</h1> + + <form action="" method="post"> + <table summary=""><tbody><tr> + <th><label for="username">Your name:</label></th> + <td> + <input type="text" id="username" name="username" /> + <span py:if="'username' in errors" class="error">${errors.username}</span> + </td> + </tr><tr> + <th><label for="url">Link URL:</label></th> + <td> + <input type="text" id="url" name="url" /> + <span py:if="'url' in errors" class="error">${errors.url}</span> + </td> + </tr><tr> + <th><label for="title">Title:</label></th> + <td> + <input type="text" name="title" /> + <span py:if="'title' in errors" class="error">${errors.title}</span> + </td> + </tr><tr> + <td></td> + <td> + <input type="submit" value="Submit" /> + <input type="submit" name="cancel" value="Cancel" /> + </td> + </tr></tbody></table> + </form> + + </body> +</html>
deleted file mode 100644 --- a/examples/webpy/README.txt +++ /dev/null @@ -1,6 +0,0 @@ -Sample application that demonstrates the use of Genshi templates in web.py -(http://webpy.org/). - -Try introducing errors (not just XML well-formedness errors, those are boring) -in the template, and you'll see a nice error screen with pretty good -information about the exact error.
deleted file mode 100644 --- a/examples/webpy/hello.html +++ /dev/null @@ -1,13 +0,0 @@ -<!DOCTYPE html - PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" - "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> -<html xmlns="http://www.w3.org/1999/xhtml" lang="en" - xmlns:py="http://genshi.edgewall.org/"> - <body> - <h1>web.py with Genshi</h1> - <hr /> - <ul py:if="times"> - <li py:for="i in range(times)">Hello, $name!</li> - </ul> - </body> -</html>
deleted file mode 100644 --- a/examples/webpy/hello.py +++ /dev/null @@ -1,27 +0,0 @@ -import os -from genshi.template import TemplateLoader -import web - -loader = TemplateLoader([os.path.dirname(os.path.abspath(__file__))], - auto_reload=True) -urls = ('/(.*)', 'hello') - - -class hello(object): - - def GET(self, name): - i = web.input(times=1) - if not name: - name = 'world' - name = name.decode('utf-8') - - tmpl = loader.load('hello.html') - stream = tmpl.generate(name=name, times=int(i.times)) - - web.header('Content-Type', 'text/html; charset=utf-8', unique=True) - print stream.render('html') - - -if __name__ == '__main__': - web.webapi.internalerror = web.debugerror - web.run(urls, globals())
--- a/genshi/__init__.py +++ b/genshi/__init__.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -21,9 +21,13 @@ __docformat__ = 'restructuredtext en' try: - __version__ = __import__('pkg_resources').get_distribution('Genshi').version + from pkg_resources import get_distribution, ResolutionError + try: + __version__ = get_distribution('Genshi').version + except ResolutionError: + __version__ = None # unknown except ImportError: - pass + __version__ = None # unknown from genshi.core import * from genshi.input import ParseError, XML, HTML
new file mode 100644 --- /dev/null +++ b/genshi/_speedups.c @@ -0,0 +1,636 @@ +/* + * Copyright (C) 2006-2008 Edgewall Software + * All rights reserved. + * + * This software is licensed as described in the file COPYING, which + * you should have received as part of this distribution. The terms + * are also available at http://genshi.edgewall.org/wiki/License. + * + * This software consists of voluntary contributions made by many + * individuals. For the exact contribution history, see the revision + * history and logs, available at http://genshi.edgewall.org/log/. + */ + +#include <Python.h> +#include <structmember.h> + +#if PY_VERSION_HEX < 0x02050000 && !defined(PY_SSIZE_T_MIN) +typedef int Py_ssize_t; +#define PY_SSIZE_T_MAX INT_MAX +#define PY_SSIZE_T_MIN INT_MIN +#endif + +static PyObject *amp1, *amp2, *lt1, *lt2, *gt1, *gt2, *qt1, *qt2; +static PyObject *stripentities, *striptags; + +static void +init_constants(void) +{ + PyObject *util = PyImport_ImportModule("genshi.util"); + stripentities = PyObject_GetAttrString(util, "stripentities"); + striptags = PyObject_GetAttrString(util, "striptags"); + Py_DECREF(util); + + amp1 = PyUnicode_DecodeASCII("&", 1, NULL); + amp2 = PyUnicode_DecodeASCII("&", 5, NULL); + lt1 = PyUnicode_DecodeASCII("<", 1, NULL); + lt2 = PyUnicode_DecodeASCII("<", 4, NULL); + gt1 = PyUnicode_DecodeASCII(">", 1, NULL); + gt2 = PyUnicode_DecodeASCII(">", 4, NULL); + qt1 = PyUnicode_DecodeASCII("\"", 1, NULL); + qt2 = PyUnicode_DecodeASCII(""", 5, NULL); +} + +/* Markup class */ + +PyTypeObject MarkupType; /* declared later */ + +PyDoc_STRVAR(Markup__doc__, +"Marks a string as being safe for inclusion in HTML/XML output without\n\ +needing to be escaped."); + +static PyObject * +escape(PyObject *text, int quotes) +{ + PyObject *args, *ret; + PyUnicodeObject *in, *out; + Py_UNICODE *inp, *outp; + int len, inn, outn; + + if (PyObject_TypeCheck(text, &MarkupType)) { + Py_INCREF(text); + return text; + } + if (PyObject_HasAttrString(text, "__html__")) { + ret = PyObject_CallMethod(text, "__html__", NULL); + args = PyTuple_New(1); + if (args == NULL) { + Py_DECREF(ret); + return NULL; + } + PyTuple_SET_ITEM(args, 0, ret); + ret = MarkupType.tp_new(&MarkupType, args, NULL); + Py_DECREF(args); + return ret; + } + in = (PyUnicodeObject *) PyObject_Unicode(text); + if (in == NULL) { + return NULL; + } + /* First we need to figure out how long the escaped string will be */ + len = inn = 0; + inp = in->str; + while (*(inp) || in->length > inp - in->str) { + switch (*inp++) { + case '&': len += 5; inn++; break; + case '"': len += quotes ? 5 : 1; inn += quotes ? 1 : 0; break; + case '<': + case '>': len += 4; inn++; break; + default: len++; + } + } + + /* Do we need to escape anything at all? */ + if (!inn) { + args = PyTuple_New(1); + if (args == NULL) { + Py_DECREF((PyObject *) in); + return NULL; + } + PyTuple_SET_ITEM(args, 0, (PyObject *) in); + ret = MarkupType.tp_new(&MarkupType, args, NULL); + Py_DECREF(args); + return ret; + } + + out = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, len); + if (out == NULL) { + Py_DECREF((PyObject *) in); + return NULL; + } + + outn = 0; + inp = in->str; + outp = out->str; + while (*(inp) || in->length > inp - in->str) { + if (outn == inn) { + /* copy rest of string if we have already replaced everything */ + Py_UNICODE_COPY(outp, inp, in->length - (inp - in->str)); + break; + } + switch (*inp) { + case '&': + Py_UNICODE_COPY(outp, ((PyUnicodeObject *) amp2)->str, 5); + outp += 5; + outn++; + break; + case '"': + if (quotes) { + Py_UNICODE_COPY(outp, ((PyUnicodeObject *) qt2)->str, 5); + outp += 5; + outn++; + } else { + *outp++ = *inp; + } + break; + case '<': + Py_UNICODE_COPY(outp, ((PyUnicodeObject *) lt2)->str, 4); + outp += 4; + outn++; + break; + case '>': + Py_UNICODE_COPY(outp, ((PyUnicodeObject *) gt2)->str, 4); + outp += 4; + outn++; + break; + default: + *outp++ = *inp; + } + inp++; + } + + Py_DECREF((PyObject *) in); + + args = PyTuple_New(1); + if (args == NULL) { + Py_DECREF((PyObject *) out); + return NULL; + } + PyTuple_SET_ITEM(args, 0, (PyObject *) out); + ret = MarkupType.tp_new(&MarkupType, args, NULL); + Py_DECREF(args); + return ret; +} + +PyDoc_STRVAR(escape__doc__, +"Create a Markup instance from a string and escape special characters\n\ +it may contain (<, >, & and \").\n\ +\n\ +>>> escape('\"1 < 2\"')\n\ +<Markup u'"1 < 2"'>\n\ +\n\ +If the `quotes` parameter is set to `False`, the \" character is left\n\ +as is. Escaping quotes is generally only required for strings that are\n\ +to be used in attribute values.\n\ +\n\ +>>> escape('\"1 < 2\"', quotes=False)\n\ +<Markup u'\"1 < 2\"'>\n\ +\n\ +:param text: the text to escape\n\ +:param quotes: if ``True``, double quote characters are escaped in\n\ + addition to the other special characters\n\ +:return: the escaped `Markup` string\n\ +:rtype: `Markup`\n\ +"); + +static PyObject * +Markup_escape(PyTypeObject* type, PyObject *args, PyObject *kwds) +{ + static char *kwlist[] = {"text", "quotes", 0}; + PyObject *text = NULL; + char quotes = 1; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|b", kwlist, &text, "es)) { + return NULL; + } + if (PyObject_Not(text)) { + return type->tp_new(type, args, NULL); + } + if (PyObject_TypeCheck(text, type)) { + Py_INCREF(text); + return text; + } + return escape(text, quotes); +} + +static PyObject * +Markup_html(PyObject *self) +{ + Py_INCREF(self); + return self; +} + +PyDoc_STRVAR(join__doc__, +"Return a `Markup` object which is the concatenation of the strings\n\ +in the given sequence, where this `Markup` object is the separator\n\ +between the joined elements.\n\ +\n\ +Any element in the sequence that is not a `Markup` instance is\n\ +automatically escaped.\n\ +\n\ +:param seq: the sequence of strings to join\n\ +:param escape_quotes: whether double quote characters in the elements\n\ + should be escaped\n\ +:return: the joined `Markup` object\n\ +:rtype: `Markup`\n\ +:see: `escape`\n\ +"); + +static PyObject * +Markup_join(PyObject *self, PyObject *args, PyObject *kwds) +{ + static char *kwlist[] = {"seq", "escape_quotes", 0}; + PyObject *seq = NULL, *seq2, *tmp, *tmp2; + char quotes = 1; + Py_ssize_t n; + int i; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|b", kwlist, &seq, "es)) { + return NULL; + } + if (!PySequence_Check(seq)) { + return NULL; + } + n = PySequence_Size(seq); + if (n < 0) { + return NULL; + } + seq2 = PyTuple_New(n); + if (seq2 == NULL) { + return NULL; + } + for (i = 0; i < n; i++) { + tmp = PySequence_GetItem(seq, i); + if (tmp == NULL) { + Py_DECREF(seq2); + return NULL; + } + tmp2 = escape(tmp, quotes); + if (tmp2 == NULL) { + Py_DECREF(seq2); + return NULL; + } + PyTuple_SET_ITEM(seq2, i, tmp2); + Py_DECREF(tmp); + } + tmp = PyUnicode_Join(self, seq2); + Py_DECREF(seq2); + if (tmp == NULL) + return NULL; + args = PyTuple_New(1); + if (args == NULL) { + Py_DECREF(tmp); + return NULL; + } + PyTuple_SET_ITEM(args, 0, tmp); + tmp = MarkupType.tp_new(&MarkupType, args, NULL); + Py_DECREF(args); + return tmp; +} + +static PyObject * +Markup_add(PyObject *self, PyObject *other) +{ + PyObject *tmp, *tmp2, *args, *ret; + if (PyObject_TypeCheck(self, &MarkupType)) { + tmp = escape(other, 1); + if (tmp == NULL) + return NULL; + tmp2 = PyUnicode_Concat(self, tmp); + } else { // __radd__ + tmp = escape(self, 1); + if (tmp == NULL) + return NULL; + tmp2 = PyUnicode_Concat(tmp, other); + } + Py_DECREF(tmp); + if (tmp2 == NULL) + return NULL; + args = PyTuple_New(1); + if (args == NULL) { + Py_DECREF(tmp2); + return NULL; + } + PyTuple_SET_ITEM(args, 0, tmp2); + ret = MarkupType.tp_new(&MarkupType, args, NULL); + Py_DECREF(args); + return ret; +} + +static PyObject * +Markup_mod(PyObject *self, PyObject *args) +{ + PyObject *tmp, *tmp2, *ret, *args2; + int i; + Py_ssize_t nargs = 0; + PyObject *kwds = NULL; + + if (PyDict_Check(args)) { + kwds = args; + } + if (kwds && PyDict_Size(kwds)) { + PyObject *kwcopy, *key, *value; + Py_ssize_t pos = 0; + + kwcopy = PyDict_Copy( kwds ); + if (kwcopy == NULL) { + return NULL; + } + while (PyDict_Next(kwcopy, &pos, &key, &value)) { + tmp = escape(value, 1); + if (tmp == NULL) { + Py_DECREF(kwcopy); + return NULL; + } + if (PyDict_SetItem(kwcopy, key, tmp) < 0) { + Py_DECREF(tmp); + Py_DECREF(kwcopy); + return NULL; + } + } + tmp = PyUnicode_Format(self, kwcopy); + Py_DECREF(kwcopy); + if (tmp == NULL) { + return NULL; + } + } else if (PyTuple_Check(args)) { + nargs = PyTuple_GET_SIZE(args); + args2 = PyTuple_New(nargs); + if (args2 == NULL) { + return NULL; + } + for (i = 0; i < nargs; i++) { + tmp = escape(PyTuple_GET_ITEM(args, i), 1); + if (tmp == NULL) { + Py_DECREF(args2); + return NULL; + } + PyTuple_SET_ITEM(args2, i, tmp); + } + tmp = PyUnicode_Format(self, args2); + Py_DECREF(args2); + if (tmp == NULL) { + return NULL; + } + } else { + tmp2 = escape(args, 1); + if (tmp2 == NULL) { + return NULL; + } + tmp = PyUnicode_Format(self, tmp2); + Py_DECREF(tmp2); + if (tmp == NULL) { + return NULL; + } + } + args = PyTuple_New(1); + if (args == NULL) { + Py_DECREF(tmp); + return NULL; + } + PyTuple_SET_ITEM(args, 0, tmp); + ret = PyUnicode_Type.tp_new(&MarkupType, args, NULL); + Py_DECREF(args); + return ret; +} + +static PyObject * +Markup_mul(PyObject *self, PyObject *num) +{ + PyObject *unicode, *result, *args; + + if (PyObject_TypeCheck(self, &MarkupType)) { + unicode = PyObject_Unicode(self); + if (unicode == NULL) return NULL; + result = PyNumber_Multiply(unicode, num); + } else { // __rmul__ + unicode = PyObject_Unicode(num); + if (unicode == NULL) return NULL; + result = PyNumber_Multiply(unicode, self); + } + Py_DECREF(unicode); + + if (result == NULL) return NULL; + args = PyTuple_New(1); + if (args == NULL) { + Py_DECREF(result); + return NULL; + } + PyTuple_SET_ITEM(args, 0, result); + result = PyUnicode_Type.tp_new(&MarkupType, args, NULL); + Py_DECREF(args); + + return result; +} + +static PyObject * +Markup_repr(PyObject *self) +{ + PyObject *format, *result, *args; + + format = PyString_FromString("<Markup %r>"); + if (format == NULL) return NULL; + result = PyObject_Unicode(self); + if (result == NULL) { + Py_DECREF(format); + return NULL; + } + args = PyTuple_New(1); + if (args == NULL) { + Py_DECREF(format); + Py_DECREF(result); + return NULL; + } + PyTuple_SET_ITEM(args, 0, result); + result = PyString_Format(format, args); + Py_DECREF(format); + Py_DECREF(args); + return result; +} + +PyDoc_STRVAR(unescape__doc__, +"Reverse-escapes &, <, >, and \" and returns a `unicode` object.\n\ +\n\ +>>> Markup('1 < 2').unescape()\n\ +u'1 < 2'\n\ +\n\ +:return: the unescaped string\n\ +:rtype: `unicode`\n\ +:see: `genshi.core.unescape`\n\ +"); + +static PyObject * +Markup_unescape(PyObject* self) +{ + PyObject *tmp, *tmp2; + + tmp = PyUnicode_Replace(self, qt2, qt1, -1); + if (tmp == NULL) return NULL; + tmp2 = PyUnicode_Replace(tmp, gt2, gt1, -1); + Py_DECREF(tmp); + if (tmp2 == NULL) return NULL; + tmp = PyUnicode_Replace(tmp2, lt2, lt1, -1); + Py_DECREF(tmp2); + if (tmp == NULL) return NULL; + tmp2 = PyUnicode_Replace(tmp, amp2, amp1, -1); + Py_DECREF(tmp); + return tmp2; +} + +PyDoc_STRVAR(stripentities__doc__, +"Return a copy of the text with any character or numeric entities\n\ +replaced by the equivalent UTF-8 characters.\n\ +\n\ +If the `keepxmlentities` parameter is provided and evaluates to `True`,\n\ +the core XML entities (``&``, ``'``, ``>``, ``<`` and\n\ +``"``) are not stripped.\n\ +\n\ +:return: a `Markup` instance with entities removed\n\ +:rtype: `Markup`\n\ +:see: `genshi.util.stripentities`\n\ +"); + +static PyObject * +Markup_stripentities(PyObject* self, PyObject *args, PyObject *kwds) +{ + static char *kwlist[] = {"keepxmlentities", 0}; + PyObject *result, *args2; + char keepxml = 0; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "|b", kwlist, &keepxml)) { + return NULL; + } + + if (stripentities == NULL) return NULL; + result = PyObject_CallFunction(stripentities, "Ob", self, keepxml); + if (result == NULL) return NULL; + args2 = PyTuple_New(1); + if (args2 == NULL) { + Py_DECREF(result); + return NULL; + } + PyTuple_SET_ITEM(args2, 0, result); + result = MarkupType.tp_new(&MarkupType, args2, NULL); + Py_DECREF(args2); + return result; +} + +PyDoc_STRVAR(striptags__doc__, +"""Return a copy of the text with all XML/HTML tags removed.\n\ +\n\ +:return: a `Markup` instance with all tags removed\n\ +:rtype: `Markup`\n\ +:see: `genshi.util.striptags`\n\ +"); + +static PyObject * +Markup_striptags(PyObject* self) +{ + PyObject *result, *args; + + if (striptags == NULL) return NULL; + result = PyObject_CallFunction(striptags, "O", self); + if (result == NULL) return NULL; + args = PyTuple_New(1); + if (args == NULL) { + Py_DECREF(result); + return NULL; + } + PyTuple_SET_ITEM(args, 0, result); + result = MarkupType.tp_new(&MarkupType, args, NULL); + Py_DECREF(args); + return result; +} + +typedef struct { + PyUnicodeObject HEAD; +} MarkupObject; + +static PyMethodDef Markup_methods[] = { + {"__html__", (PyCFunction) Markup_html, METH_NOARGS, NULL}, + {"escape", (PyCFunction) Markup_escape, + METH_VARARGS|METH_CLASS|METH_KEYWORDS, escape__doc__}, + {"join", (PyCFunction)Markup_join, METH_VARARGS|METH_KEYWORDS, join__doc__}, + {"unescape", (PyCFunction)Markup_unescape, METH_NOARGS, unescape__doc__}, + {"stripentities", (PyCFunction) Markup_stripentities, + METH_VARARGS|METH_KEYWORDS, stripentities__doc__}, + {"striptags", (PyCFunction) Markup_striptags, METH_NOARGS, + striptags__doc__}, + {NULL} /* Sentinel */ +}; + +static PyNumberMethods Markup_as_number = { + Markup_add, /*nb_add*/ + 0, /*nb_subtract*/ + Markup_mul, /*nb_multiply*/ + 0, /*nb_divide*/ + Markup_mod, /*nb_remainder*/ +}; + +PyTypeObject MarkupType = { + PyObject_HEAD_INIT(NULL) + 0, + "genshi._speedups.Markup", + sizeof(MarkupObject), + 0, + 0, /*tp_dealloc*/ + 0, /*tp_print*/ + 0, /*tp_getattr*/ + 0, /*tp_setattr*/ + 0, /*tp_compare*/ + Markup_repr, /*tp_repr*/ + &Markup_as_number, /*tp_as_number*/ + 0, /*tp_as_sequence*/ + 0, /*tp_as_mapping*/ + 0, /*tp_hash */ + + 0, /*tp_call*/ + 0, /*tp_str*/ + 0, /*tp_getattro*/ + 0, /*tp_setattro*/ + 0, /*tp_as_buffer*/ + + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_CHECKTYPES, /*tp_flags*/ + Markup__doc__,/*tp_doc*/ + + 0, /*tp_traverse*/ + 0, /*tp_clear*/ + + 0, /*tp_richcompare*/ + 0, /*tp_weaklistoffset*/ + + 0, /*tp_iter*/ + 0, /*tp_iternext*/ + + /* Attribute descriptor and subclassing stuff */ + + Markup_methods,/*tp_methods*/ + 0, /*tp_members*/ + 0, /*tp_getset*/ + 0, /*tp_base*/ + 0, /*tp_dict*/ + + 0, /*tp_descr_get*/ + 0, /*tp_descr_set*/ + 0, /*tp_dictoffset*/ + + 0, /*tp_init*/ + 0, /*tp_alloc will be set to PyType_GenericAlloc in module init*/ + 0, /*tp_new*/ + 0, /*tp_free Low-level free-memory routine */ + 0, /*tp_is_gc For PyObject_IS_GC */ + 0, /*tp_bases*/ + 0, /*tp_mro method resolution order */ + 0, /*tp_cache*/ + 0, /*tp_subclasses*/ + 0 /*tp_weaklist*/ +}; + +PyMODINIT_FUNC +init_speedups(void) +{ + PyObject *module; + + /* Workaround for quirk in Visual Studio, see + <http://www.python.it/faq/faq-3.html#3.24> */ + MarkupType.tp_base = &PyUnicode_Type; + + if (PyType_Ready(&MarkupType) < 0) + return; + + init_constants(); + + module = Py_InitModule("_speedups", NULL); + Py_INCREF(&MarkupType); + PyModule_AddObject(module, "Markup", (PyObject *) &MarkupType); +}
--- a/genshi/builder.py +++ b/genshi/builder.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -68,7 +68,8 @@ Hello, <em>world</em>! """ -from genshi.core import Attrs, Namespace, QName, Stream, START, END, TEXT +from genshi.core import Attrs, Markup, Namespace, QName, Stream, \ + START, END, TEXT __all__ = ['Fragment', 'Element', 'ElementFactory', 'tag'] __docformat__ = 'restructuredtext en' @@ -107,6 +108,9 @@ def __unicode__(self): return unicode(self.generate()) + def __html__(self): + return Markup(self.generate()) + def append(self, node): """Append an element or string as child node. @@ -146,14 +150,15 @@ return Stream(self._generate()) -def _value_to_unicode(value): - if isinstance(value, unicode): - return value - return unicode(value) - def _kwargs_to_attrs(kwargs): - return [(QName(k.rstrip('_').replace('_', '-')), _value_to_unicode(v)) - for k, v in kwargs.items() if v is not None] + attrs = [] + names = set() + for name, value in kwargs.items(): + name = name.rstrip('_').replace('_', '-') + if value is not None and name not in names: + attrs.append((QName(name), unicode(value))) + names.add(name) + return Attrs(attrs) class Element(Fragment): @@ -240,7 +245,7 @@ def __init__(self, tag_, **attrib): Fragment.__init__(self) self.tag = QName(tag_) - self.attrib = Attrs(_kwargs_to_attrs(attrib)) + self.attrib = _kwargs_to_attrs(attrib) def __call__(self, *args, **kwargs): """Append any positional arguments as child nodes, and keyword arguments @@ -250,7 +255,7 @@ :rtype: `Element` :see: `Fragment.append` """ - self.attrib |= Attrs(_kwargs_to_attrs(kwargs)) + self.attrib |= _kwargs_to_attrs(kwargs) Fragment.__call__(self, *args) return self
--- a/genshi/core.py +++ b/genshi/core.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -13,6 +13,11 @@ """Core classes for markup processing.""" +try: + from functools import reduce +except ImportError: + pass # builtin in Python <= 2.5 +from itertools import chain import operator from genshi.util import plaintext, stripentities, striptags @@ -51,7 +56,7 @@ returns the complete generated text at once. Both accept various parameters that impact the way the stream is serialized. """ - __slots__ = ['events'] + __slots__ = ['events', 'serializer'] START = StreamEventKind('START') #: a start tag END = StreamEventKind('END') #: an end tag @@ -65,12 +70,17 @@ PI = StreamEventKind('PI') #: processing instruction COMMENT = StreamEventKind('COMMENT') #: comment - def __init__(self, events): + def __init__(self, events, serializer=None): """Initialize the stream with a sequence of markup events. :param events: a sequence or iterable providing the events + :param serializer: the default serialization method to use for this + stream + + :note: Changed in 0.5: added the `serializer` argument """ self.events = events #: The underlying iterable producing the events + self.serializer = serializer #: The default serializion method def __iter__(self): return iter(self.events) @@ -119,7 +129,7 @@ :return: the filtered stream :rtype: `Stream` """ - return Stream(_ensure(function(self))) + return Stream(_ensure(function(self)), serializer=self.serializer) def filter(self, *filters): """Apply filters to the stream. @@ -143,7 +153,7 @@ """ return reduce(operator.or_, (self,) + filters) - def render(self, method='xml', encoding='utf-8', **kwargs): + def render(self, method=None, encoding='utf-8', out=None, **kwargs): """Return a string representation of the stream. Any additional keyword arguments are passed to the serializer, and thus @@ -151,21 +161,52 @@ :param method: determines how the stream is serialized; can be either "xml", "xhtml", "html", "text", or a custom serializer - class + class; if `None`, the default serialization method of + the stream is used :param encoding: how the output string should be encoded; if set to `None`, this method returns a `unicode` object - :return: a `str` or `unicode` object + :param out: a file-like object that the output should be written to + instead of being returned as one big string; note that if + this is a file or socket (or similar), the `encoding` must + not be `None` (that is, the output must be encoded) + :return: a `str` or `unicode` object (depending on the `encoding` + parameter), or `None` if the `out` parameter is provided :rtype: `basestring` + :see: XMLSerializer, XHTMLSerializer, HTMLSerializer, TextSerializer + :note: Changed in 0.5: added the `out` parameter """ from genshi.output import encode + if method is None: + method = self.serializer or 'xml' generator = self.serialize(method=method, **kwargs) - return encode(generator, method=method, encoding=encoding) + return encode(generator, method=method, encoding=encoding, out=out) def select(self, path, namespaces=None, variables=None): """Return a new stream that contains the events matching the given XPath expression. + >>> from genshi import HTML + >>> stream = HTML('<doc><elem>foo</elem><elem>bar</elem></doc>') + >>> print stream.select('elem') + <elem>foo</elem><elem>bar</elem> + >>> print stream.select('elem/text()') + foobar + + Note that the outermost element of the stream becomes the *context + node* for the XPath test. That means that the expression "doc" would + not match anything in the example above, because it only tests against + child elements of the outermost element: + + >>> print stream.select('doc') + <BLANKLINE> + + You can use the "." expression to match the context node itself + (although that usually makes little sense): + + >>> print stream.select('.') + <doc><elem>foo</elem><elem>bar</elem></doc> + :param path: a string containing the XPath expression :param namespaces: mapping of namespace prefixes used in the path :param variables: mapping of variable names to values @@ -190,13 +231,16 @@ :param method: determines how the stream is serialized; can be either "xml", "xhtml", "html", "text", or a custom serializer - class + class; if `None`, the default serialization method of + the stream is used :return: an iterator over the serialization results (`Markup` or `unicode` objects, depending on the serialization method) :rtype: ``iterator`` :see: XMLSerializer, XHTMLSerializer, HTMLSerializer, TextSerializer """ from genshi.output import get_serializer + if method is None: + method = self.serializer or 'xml' return get_serializer(method, **kwargs)(_ensure(self)) def __str__(self): @@ -205,6 +249,9 @@ def __unicode__(self): return self.render(encoding=None) + def __html__(self): + return self + START = Stream.START END = Stream.END @@ -220,12 +267,24 @@ def _ensure(stream): """Ensure that every item on the stream is actually a markup event.""" - for event in stream: - if type(event) is not tuple: + stream = iter(stream) + event = stream.next() + + # Check whether the iterable is a real markup event stream by examining the + # first item it yields; if it's not we'll need to do some conversion + if type(event) is not tuple or len(event) != 3: + for event in chain([event], stream): if hasattr(event, 'totuple'): event = event.totuple() else: event = TEXT, unicode(event), (None, -1, -1) + yield event + return + + # This looks like a markup event stream, so we'll just pass it through + # unchanged + yield event + for event in stream: yield event @@ -295,6 +354,12 @@ return True def __getslice__(self, i, j): + """Return a slice of the attributes list. + + >>> attrs = Attrs([('href', '#'), ('title', 'Foo')]) + >>> attrs[1:] + Attrs([('title', 'Foo')]) + """ return Attrs(tuple.__getslice__(self, i, j)) def __or__(self, attrs): @@ -361,11 +426,6 @@ """ __slots__ = [] - def __new__(cls, text='', *args): - if args: - text %= tuple(map(escape, args)) - return unicode.__new__(cls, text) - def __add__(self, other): return Markup(unicode(self) + unicode(escape(other))) @@ -373,9 +433,13 @@ return Markup(unicode(escape(other)) + unicode(self)) def __mod__(self, args): - if not isinstance(args, (list, tuple)): - args = [args] - return Markup(unicode.__mod__(self, tuple(map(escape, args)))) + if isinstance(args, dict): + args = dict(zip(args.keys(), map(escape, args.values()))) + elif isinstance(args, (list, tuple)): + args = tuple(map(escape, args)) + else: + args = escape(args) + return Markup(unicode.__mod__(self, args)) def __mul__(self, num): return Markup(unicode(self) * num) @@ -428,6 +492,9 @@ return cls() if type(text) is cls: return text + if hasattr(text, '__html__'): + return Markup(text.__html__()) + text = unicode(text).replace('&', '&') \ .replace('<', '<') \ .replace('>', '>') @@ -477,6 +544,11 @@ return Markup(striptags(self)) +try: + from genshi._speedups import Markup +except ImportError: + pass # just use the Python implementation + escape = Markup.escape def unescape(text): @@ -543,7 +615,7 @@ def __new__(cls, uri): if type(uri) is cls: return uri - return object.__new__(cls, uri) + return object.__new__(cls) def __getnewargs__(self): return (self.uri,) @@ -572,6 +644,9 @@ return QName(self.uri + u'}' + name) __getattr__ = __getitem__ + def __hash__(self): + return hash(self.uri) + def __repr__(self): return '<Namespace "%s">' % self.uri @@ -590,9 +665,9 @@ """A qualified element or attribute name. The unicode value of instances of this class contains the qualified name of - the element or attribute, in the form ``{namespace}localname``. The namespace - URI can be obtained through the additional `namespace` attribute, while the - local name can be accessed through the `localname` attribute. + the element or attribute, in the form ``{namespace-uri}local-name``. The + namespace URI can be obtained through the additional `namespace` attribute, + while the local name can be accessed through the `localname` attribute. >>> qname = QName('foo') >>> qname @@ -612,10 +687,16 @@ __slots__ = ['namespace', 'localname'] def __new__(cls, qname): + """Create the `QName` instance. + + :param qname: the qualified name as a string of the form + ``{namespace-uri}local-name``, where the leading curly + brace is optional + """ if type(qname) is cls: return qname - parts = qname.split(u'}', 1) + parts = qname.lstrip(u'{').split(u'}', 1) if len(parts) > 1: self = unicode.__new__(cls, u'{%s' % qname) self.namespace, self.localname = map(unicode, parts)
--- a/genshi/filters/__init__.py +++ b/genshi/filters/__init__.py @@ -15,5 +15,6 @@ from genshi.filters.html import HTMLFormFiller, HTMLSanitizer from genshi.filters.i18n import Translator +from genshi.filters.transform import Transformer __docformat__ = 'restructuredtext en'
--- a/genshi/filters/html.py +++ b/genshi/filters/html.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -13,14 +13,10 @@ """Implementation of a number of stream filters.""" -try: - frozenset -except NameError: - from sets import ImmutableSet as frozenset import re from genshi.core import Attrs, QName, stripentities -from genshi.core import END, START, TEXT +from genshi.core import END, START, TEXT, COMMENT __all__ = ['HTMLFormFiller', 'HTMLSanitizer'] __docformat__ = 'restructuredtext en' @@ -69,7 +65,9 @@ """ in_form = in_select = in_option = in_textarea = False select_value = option_value = textarea_value = None - option_start = option_text = None + option_start = None + option_text = [] + no_option_value = False for kind, data, pos in stream: @@ -94,13 +92,13 @@ checked = False if isinstance(value, (list, tuple)): if declval: - checked = declval in [str(v) for v + checked = declval in [unicode(v) for v in value] else: checked = bool(filter(None, value)) else: if declval: - checked = declval == str(value) + checked = declval == unicode(value) elif type == 'checkbox': checked = bool(value) if checked: @@ -130,15 +128,18 @@ elif in_select and tagname == 'option': option_start = kind, data, pos option_value = attrs.get('value') + if option_value is None: + no_option_value = True + option_value = '' in_option = True continue yield kind, (tag, attrs), pos elif in_form and kind is TEXT: if in_select and in_option: - if option_value is None: - option_value = data - option_text = kind, data, pos + if no_option_value: + option_value += data + option_text.append((kind, data, pos)) continue elif in_textarea: continue @@ -153,10 +154,10 @@ select_value = None elif in_select and tagname == 'option': if isinstance(select_value, (tuple, list)): - selected = option_value in [str(v) for v + selected = option_value in [unicode(v) for v in select_value] else: - selected = option_value == str(select_value) + selected = option_value == unicode(select_value) okind, (tag, attrs), opos = option_start if selected: attrs |= [(QName('selected'), 'selected')] @@ -164,9 +165,12 @@ attrs -= 'selected' yield okind, (tag, attrs), opos if option_text: - yield option_text + for event in option_text: + yield event in_option = False - option_start = option_text = option_value = None + no_option_value = False + option_start = option_value = None + option_text = [] elif tagname == 'textarea': if textarea_value: yield TEXT, unicode(textarea_value), pos @@ -206,6 +210,9 @@ well as a lot of other things. However, the style tag is still excluded by default because it is very hard for such sanitizing to be completely safe, especially considering how much error recovery current web browsers perform. + + :warn: Note that this special processing of CSS is currently only applied to + style attributes, **not** style elements. """ SAFE_TAGS = frozenset(['a', 'abbr', 'acronym', 'address', 'area', 'b', @@ -247,9 +254,13 @@ :param uri_attrs: a set of names of attributes that contain URIs """ self.safe_tags = safe_tags + "The set of tag names that are considered safe." self.safe_attrs = safe_attrs + "The set of attribute names that are considered safe." self.uri_attrs = uri_attrs + "The set of names of attributes that may contain URIs." self.safe_schemes = safe_schemes + "The set of URI schemes that are considered safe." def __call__(self, stream): """Apply the filter to the given stream. @@ -258,12 +269,6 @@ """ waiting_for = None - def _get_scheme(href): - if ':' not in href: - return None - chars = [char for char in href.split(':', 1)[0] if char.isalnum()] - return ''.join(chars).lower() - for kind, data, pos in stream: if kind is START: if waiting_for: @@ -280,22 +285,11 @@ continue elif attr in self.uri_attrs: # Don't allow URI schemes such as "javascript:" - if _get_scheme(value) not in self.safe_schemes: + if not self.is_safe_uri(value): continue elif attr == 'style': # Remove dangerous CSS declarations from inline styles - decls = [] - value = self._replace_unicode_escapes(value) - for decl in filter(None, value.split(';')): - is_evil = False - if 'expression' in decl: - is_evil = True - for m in re.finditer(r'url\s*\(([^)]+)', decl): - if _get_scheme(m.group(1)) not in self.safe_schemes: - is_evil = True - break - if not is_evil: - decls.append(decl.strip()) + decls = self.sanitize_css(value) if not decls: continue value = '; '.join(decls) @@ -311,10 +305,79 @@ else: yield kind, data, pos - else: + elif kind is not COMMENT: if not waiting_for: yield kind, data, pos + def is_safe_uri(self, uri): + """Determine whether the given URI is to be considered safe for + inclusion in the output. + + The default implementation checks whether the scheme of the URI is in + the set of allowed URIs (`safe_schemes`). + + >>> sanitizer = HTMLSanitizer() + >>> sanitizer.is_safe_uri('http://example.org/') + True + >>> sanitizer.is_safe_uri('javascript:alert(document.cookie)') + False + + :param uri: the URI to check + :return: `True` if the URI can be considered safe, `False` otherwise + :rtype: `bool` + :since: version 0.4.3 + """ + if ':' not in uri: + return True # This is a relative URI + chars = [char for char in uri.split(':', 1)[0] if char.isalnum()] + return ''.join(chars).lower() in self.safe_schemes + + def sanitize_css(self, text): + """Remove potentially dangerous property declarations from CSS code. + + In particular, properties using the CSS ``url()`` function with a scheme + that is not considered safe are removed: + + >>> sanitizer = HTMLSanitizer() + >>> sanitizer.sanitize_css(u''' + ... background: url(javascript:alert("foo")); + ... color: #000; + ... ''') + [u'color: #000'] + + Also, the proprietary Internet Explorer function ``expression()`` is + always stripped: + + >>> sanitizer.sanitize_css(u''' + ... background: #fff; + ... color: #000; + ... width: e/**/xpression(alert("foo")); + ... ''') + [u'background: #fff', u'color: #000'] + + :param text: the CSS text; this is expected to be `unicode` and to not + contain any character or numeric references + :return: a list of declarations that are considered safe + :rtype: `list` + :since: version 0.4.3 + """ + decls = [] + text = self._strip_css_comments(self._replace_unicode_escapes(text)) + for decl in filter(None, text.split(';')): + decl = decl.strip() + if not decl: + continue + is_evil = False + if 'expression' in decl: + is_evil = True + for match in re.finditer(r'url\s*\(([^)]+)', decl): + if not self.is_safe_uri(match.group(1)): + is_evil = True + break + if not is_evil: + decls.append(decl.strip()) + return decls + _NORMALIZE_NEWLINES = re.compile(r'\r\n').sub _UNICODE_ESCAPE = re.compile(r'\\([0-9a-fA-F]{1,6})\s?').sub @@ -322,3 +385,8 @@ def _repl(match): return unichr(int(match.group(1), 16)) return self._UNICODE_ESCAPE(_repl, self._NORMALIZE_NEWLINES('\n', text)) + + _CSS_COMMENTS = re.compile(r'/\*.*?\*/').sub + + def _strip_css_comments(self, text): + return self._CSS_COMMENTS('', text)
--- a/genshi/filters/i18n.py +++ b/genshi/filters/i18n.py @@ -1,24 +1,74 @@ -"""Utilities for internationalization and localization of templates.""" +# -*- coding: utf-8 -*- +# +# Copyright (C) 2007 Edgewall Software +# All rights reserved. +# +# This software is licensed as described in the file COPYING, which +# you should have received as part of this distribution. The terms +# are also available at http://genshi.edgewall.org/wiki/License. +# +# This software consists of voluntary contributions made by many +# individuals. For the exact contribution history, see the revision +# history and logs, available at http://genshi.edgewall.org/log/. -try: - frozenset -except NameError: - from sets import ImmutableSet as frozenset -from gettext import gettext -from opcode import opmap +"""Utilities for internationalization and localization of templates. + +:since: version 0.4 +""" + +from gettext import NullTranslations import re +from types import FunctionType -from genshi.core import Attrs, Namespace, QName, START, END, TEXT, _ensure -from genshi.template.base import Template, EXPR, SUB -from genshi.template.markup import EXEC +from genshi.core import Attrs, Namespace, QName, START, END, TEXT, START_NS, \ + END_NS, XML_NAMESPACE, _ensure +from genshi.template.eval import _ast +from genshi.template.base import DirectiveFactory, EXPR, SUB, _apply_directives +from genshi.template.directives import Directive +from genshi.template.markup import MarkupTemplate, EXEC -_LOAD_NAME = chr(opmap['LOAD_NAME']) -_LOAD_CONST = chr(opmap['LOAD_CONST']) -_CALL_FUNCTION = chr(opmap['CALL_FUNCTION']) -_BINARY_ADD = chr(opmap['BINARY_ADD']) +__all__ = ['Translator', 'extract'] +__docformat__ = 'restructuredtext en' + +I18N_NAMESPACE = Namespace('http://genshi.edgewall.org/i18n') -class Translator(object): +class CommentDirective(Directive): + + __slots__ = [] + + @classmethod + def attach(cls, template, stream, value, namespaces, pos): + return None, stream + + +class MsgDirective(Directive): + + __slots__ = ['params'] + + def __init__(self, value, template, hints=None, namespaces=None, + lineno=-1, offset=-1): + Directive.__init__(self, None, template, namespaces, lineno, offset) + self.params = [name.strip() for name in value.split(',')] + + def __call__(self, stream, directives, ctxt, **vars): + msgbuf = MessageBuffer(self.params) + + stream = iter(stream) + yield stream.next() # the outer start tag + previous = stream.next() + for event in stream: + msgbuf.append(*previous) + previous = event + + gettext = ctxt.get('_i18n.gettext') + for event in msgbuf.translate(gettext(msgbuf.format())): + yield event + + yield previous # the outer end tag + + +class Translator(DirectiveFactory): """Can extract and translate localizable strings from markup streams and templates. @@ -65,27 +115,45 @@ <p>Hallo, Hans</p> </body> </html> + + Note that elements defining ``xml:lang`` attributes that do not contain + variable expressions are ignored by this filter. That can be used to + exclude specific parts of a template from being extracted and translated. """ + directives = [ + ('comment', CommentDirective), + ('msg', MsgDirective) + ] + IGNORE_TAGS = frozenset([ QName('script'), QName('http://www.w3.org/1999/xhtml}script'), QName('style'), QName('http://www.w3.org/1999/xhtml}style') ]) INCLUDE_ATTRS = frozenset(['abbr', 'alt', 'label', 'prompt', 'standby', 'summary', 'title']) + NAMESPACE = I18N_NAMESPACE - def __init__(self, translate=gettext, ignore_tags=IGNORE_TAGS, - include_attrs=INCLUDE_ATTRS): + def __init__(self, translate=NullTranslations(), ignore_tags=IGNORE_TAGS, + include_attrs=INCLUDE_ATTRS, extract_text=True): """Initialize the translator. :param translate: the translation function, for example ``gettext`` or ``ugettext``. :param ignore_tags: a set of tag names that should not be localized :param include_attrs: a set of attribute names should be localized + :param extract_text: whether the content of text nodes should be + extracted, or only text in explicit ``gettext`` + function calls + + :note: Changed in 0.6: the `translate` parameter can now be either + a ``gettext``-style function, or an object compatible with the + ``NullTransalations`` or ``GNUTranslations`` interface """ self.translate = translate self.ignore_tags = ignore_tags self.include_attrs = include_attrs + self.extract_text = extract_text def __call__(self, stream, ctxt=None, search_text=True): """Translate any localizable strings in the given stream. @@ -104,27 +172,36 @@ """ ignore_tags = self.ignore_tags include_attrs = self.include_attrs - translate = self.translate skip = 0 + xml_lang = XML_NAMESPACE['lang'] + + if type(self.translate) is FunctionType: + gettext = self.translate + else: + gettext = self.translate.ugettext + if ctxt: + ctxt['_i18n.gettext'] = gettext + + extract_text = self.extract_text + if not extract_text: + search_text = False for kind, data, pos in stream: # skip chunks that should not be localized if skip: if kind is START: - tag, attrs = data - if tag in ignore_tags: - skip += 1 + skip += 1 elif kind is END: - if tag in ignore_tags: - skip -= 1 + skip -= 1 yield kind, data, pos continue # handle different events that can be localized if kind is START: tag, attrs = data - if tag in ignore_tags: + if tag in self.ignore_tags or \ + isinstance(attrs.get(xml_lang), basestring): skip += 1 yield kind, data, pos continue @@ -133,32 +210,37 @@ changed = False for name, value in attrs: newval = value - if isinstance(value, basestring): + if extract_text and isinstance(value, basestring): if name in include_attrs: - newval = self.translate(value) + newval = gettext(value) else: newval = list(self(_ensure(value), ctxt, - search_text=name in include_attrs) + search_text=False) ) if newval != value: value = newval changed = True new_attrs.append((name, value)) if changed: - attrs = new_attrs + attrs = Attrs(new_attrs) yield kind, (tag, attrs), pos elif search_text and kind is TEXT: text = data.strip() if text: - data = data.replace(text, translate(text)) + data = data.replace(text, unicode(gettext(text))) yield kind, data, pos elif kind is SUB: - subkind, substream = data - new_substream = list(self(substream, ctxt)) - yield kind, (subkind, new_substream), pos + directives, substream = data + # If this is an i18n:msg directive, no need to translate text + # nodes here + is_msg = filter(None, [isinstance(d, MsgDirective) + for d in directives]) + substream = list(self(substream, ctxt, + search_text=not is_msg)) + yield kind, (directives, substream), pos else: yield kind, data, pos @@ -167,17 +249,20 @@ 'ugettext', 'ungettext') def extract(self, stream, gettext_functions=GETTEXT_FUNCTIONS, - search_text=True): + search_text=True, msgbuf=None): """Extract localizable strings from the given template stream. For every string found, this function yields a ``(lineno, function, - message)`` tuple, where: + message, comments)`` tuple, where: * ``lineno`` is the number of the line on which the string was found, * ``function`` is the name of the ``gettext`` function used (if the string was extracted from embedded Python code), and * ``message`` is the string itself (a ``unicode`` object, or a tuple - of ``unicode`` objects for functions with multiple string arguments). + of ``unicode`` objects for functions with multiple string + arguments). + * ``comments`` is a list of comments related to the message, extracted + from ``i18n:comment`` attributes found in the markup >>> from genshi.template import MarkupTemplate >>> @@ -192,12 +277,12 @@ ... </body> ... </html>''', filename='example.html') >>> - >>> for lineno, funcname, message in Translator().extract(tmpl.stream): - ... print "%d, %r, %r" % (lineno, funcname, message) + >>> for line, func, msg, comments in Translator().extract(tmpl.stream): + ... print "%d, %r, %r" % (line, func, msg) 3, None, u'Example' 6, None, u'Example' 7, '_', u'Hello, %(name)s' - 8, 'ngettext', (u'You have %d item', u'You have %d items') + 8, 'ngettext', (u'You have %d item', u'You have %d items', None) :param stream: the event stream to extract strings from; can be a regular stream or a template stream @@ -210,77 +295,302 @@ :note: Changed in 0.4.1: For a function with multiple string arguments (such as ``ngettext``), a single item with a tuple of strings is yielded, instead an item for each string argument. + :note: Changed in 0.6: The returned tuples now include a 4th element, + which is a list of comments for the translator """ - tagname = None + if not self.extract_text: + search_text = False skip = 0 + i18n_comment = I18N_NAMESPACE['comment'] + i18n_msg = I18N_NAMESPACE['msg'] + xml_lang = XML_NAMESPACE['lang'] for kind, data, pos in stream: + if skip: if kind is START: - tag, attrs = data - if tag in self.ignore_tags: - skip += 1 + skip += 1 if kind is END: - tag = data - if tag in self.ignore_tags: - skip -= 1 - continue + skip -= 1 - if kind is START: + if kind is START and not skip: tag, attrs = data - if tag in self.ignore_tags: + + if tag in self.ignore_tags or \ + isinstance(attrs.get(xml_lang), basestring): skip += 1 continue for name, value in attrs: - if isinstance(value, basestring): + if search_text and isinstance(value, basestring): if name in self.include_attrs: text = value.strip() if text: - yield pos[1], None, text + yield pos[1], None, text, [] else: - for lineno, funcname, text in self.extract( + for lineno, funcname, text, comments in self.extract( _ensure(value), gettext_functions, - search_text=name in self.include_attrs): - yield lineno, funcname, text + search_text=False): + yield lineno, funcname, text, comments - elif search_text and kind is TEXT: - text = data.strip() - if text and filter(None, [ch.isalpha() for ch in text]): - yield pos[1], None, text + if msgbuf: + msgbuf.append(kind, data, pos) + else: + msg_params = attrs.get(i18n_msg) + if msg_params is not None: + if type(msg_params) is list: # event tuple + msg_params = msg_params[0][1] + msgbuf = MessageBuffer( + msg_params, attrs.get(i18n_comment), pos[1] + ) + + elif not skip and search_text and kind is TEXT: + if not msgbuf: + text = data.strip() + if text and filter(None, [ch.isalpha() for ch in text]): + yield pos[1], None, text, [] + else: + msgbuf.append(kind, data, pos) + + elif not skip and msgbuf and kind is END: + msgbuf.append(kind, data, pos) + if not msgbuf.depth: + yield msgbuf.lineno, None, msgbuf.format(), \ + filter(None, [msgbuf.comment]) + msgbuf = None elif kind is EXPR or kind is EXEC: - consts = dict([(n, chr(i) + '\x00') for i, n in - enumerate(data.code.co_consts)]) - gettext_locs = [consts[n] for n in gettext_functions - if n in consts] - ops = [ - _LOAD_CONST, '(', '|'.join(gettext_locs), ')', - _CALL_FUNCTION, '.\x00', - '((?:', _BINARY_ADD, '|', _LOAD_CONST, '.\x00)+)' - ] - for loc, opcodes in re.findall(''.join(ops), data.code.co_code): - funcname = data.code.co_consts[ord(loc[0])] - strings = [] - opcodes = iter(opcodes) - for opcode in opcodes: - if opcode == _BINARY_ADD: - arg = strings.pop() - strings[-1] += arg - else: - arg = data.code.co_consts[ord(opcodes.next())] - opcodes.next() # skip second byte - if not isinstance(arg, basestring): - break - strings.append(unicode(arg)) - if len(strings) == 1: - strings = strings[0] - else: - strings = tuple(strings) - yield pos[1], funcname, strings + if msgbuf: + msgbuf.append(kind, data, pos) + for funcname, strings in extract_from_code(data, + gettext_functions): + yield pos[1], funcname, strings, [] elif kind is SUB: subkind, substream = data - for lineno, funcname, text in self.extract(substream, - gettext_functions): - yield lineno, funcname, text + messages = self.extract(substream, gettext_functions, + search_text=search_text and not skip, + msgbuf=msgbuf) + for lineno, funcname, text, comments in messages: + yield lineno, funcname, text, comments + + +class MessageBuffer(object): + """Helper class for managing internationalized mixed content. + + :since: version 0.5 + """ + + def __init__(self, params=u'', comment=None, lineno=-1): + """Initialize the message buffer. + + :param params: comma-separated list of parameter names + :type params: `basestring` + :param lineno: the line number on which the first stream event + belonging to the message was found + """ + if isinstance(params, basestring): + params = [name.strip() for name in params.split(',')] + self.params = params + self.comment = comment + self.lineno = lineno + self.string = [] + self.events = {} + self.values = {} + self.depth = 1 + self.order = 1 + self.stack = [0] + + def append(self, kind, data, pos): + """Append a stream event to the buffer. + + :param kind: the stream event kind + :param data: the event data + :param pos: the position of the event in the source + """ + if kind is TEXT: + self.string.append(data) + self.events.setdefault(self.stack[-1], []).append(None) + elif kind is EXPR: + param = self.params.pop(0) + self.string.append('%%(%s)s' % param) + self.events.setdefault(self.stack[-1], []).append(None) + self.values[param] = (kind, data, pos) + else: + if kind is START: + self.string.append(u'[%d:' % self.order) + self.events.setdefault(self.order, []).append((kind, data, pos)) + self.stack.append(self.order) + self.depth += 1 + self.order += 1 + elif kind is END: + self.depth -= 1 + if self.depth: + self.events[self.stack[-1]].append((kind, data, pos)) + self.string.append(u']') + self.stack.pop() + + def format(self): + """Return a message identifier representing the content in the + buffer. + """ + return u''.join(self.string).strip() + + def translate(self, string, regex=re.compile(r'%\((\w+)\)s')): + """Interpolate the given message translation with the events in the + buffer and return the translated stream. + + :param string: the translated message string + """ + parts = parse_msg(string) + for order, string in parts: + events = self.events[order] + while events: + event = events.pop(0) + if event: + yield event + else: + if not string: + break + for idx, part in enumerate(regex.split(string)): + if idx % 2: + yield self.values[part] + elif part: + yield TEXT, part, (None, -1, -1) + if not self.events[order] or not self.events[order][0]: + break + + +def parse_msg(string, regex=re.compile(r'(?:\[(\d+)\:)|\]')): + """Parse a translated message using Genshi mixed content message + formatting. + + >>> parse_msg("See [1:Help].") + [(0, 'See '), (1, 'Help'), (0, '.')] + + >>> parse_msg("See [1:our [2:Help] page] for details.") + [(0, 'See '), (1, 'our '), (2, 'Help'), (1, ' page'), (0, ' for details.')] + + >>> parse_msg("[2:Details] finden Sie in [1:Hilfe].") + [(2, 'Details'), (0, ' finden Sie in '), (1, 'Hilfe'), (0, '.')] + + >>> parse_msg("[1:] Bilder pro Seite anzeigen.") + [(1, ''), (0, ' Bilder pro Seite anzeigen.')] + + :param string: the translated message string + :return: a list of ``(order, string)`` tuples + :rtype: `list` + """ + parts = [] + stack = [0] + while True: + mo = regex.search(string) + if not mo: + break + + if mo.start() or stack[-1]: + parts.append((stack[-1], string[:mo.start()])) + string = string[mo.end():] + + orderno = mo.group(1) + if orderno is not None: + stack.append(int(orderno)) + else: + stack.pop() + if not stack: + break + + if string: + parts.append((stack[-1], string)) + + return parts + + +def extract_from_code(code, gettext_functions): + """Extract strings from Python bytecode. + + >>> from genshi.template.eval import Expression + + >>> expr = Expression('_("Hello")') + >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS)) + [('_', u'Hello')] + + >>> expr = Expression('ngettext("You have %(num)s item", ' + ... '"You have %(num)s items", num)') + >>> list(extract_from_code(expr, Translator.GETTEXT_FUNCTIONS)) + [('ngettext', (u'You have %(num)s item', u'You have %(num)s items', None))] + + :param code: the `Code` object + :type code: `genshi.template.eval.Code` + :param gettext_functions: a sequence of function names + :since: version 0.5 + """ + def _walk(node): + if isinstance(node, _ast.Call) and isinstance(node.func, _ast.Name) \ + and node.func.id in gettext_functions: + strings = [] + def _add(arg): + if isinstance(arg, _ast.Str) and isinstance(arg.s, basestring): + strings.append(unicode(arg.s, 'utf-8')) + elif arg: + strings.append(None) + [_add(arg) for arg in node.args] + _add(node.starargs) + _add(node.kwargs) + if len(strings) == 1: + strings = strings[0] + else: + strings = tuple(strings) + yield node.func.id, strings + elif node._fields: + children = [] + for field in node._fields: + child = getattr(node, field, None) + if isinstance(child, list): + for elem in child: + children.append(elem) + elif isinstance(child, _ast.AST): + children.append(child) + for child in children: + for funcname, strings in _walk(child): + yield funcname, strings + return _walk(code.ast) + + +def extract(fileobj, keywords, comment_tags, options): + """Babel extraction method for Genshi templates. + + :param fileobj: the file-like object the messages should be extracted from + :param keywords: a list of keywords (i.e. function names) that should be + recognized as translation functions + :param comment_tags: a list of translator tags to search for and include + in the results + :param options: a dictionary of additional options (optional) + :return: an iterator over ``(lineno, funcname, message, comments)`` tuples + :rtype: ``iterator`` + """ + template_class = options.get('template_class', MarkupTemplate) + if isinstance(template_class, basestring): + module, clsname = template_class.split(':', 1) + template_class = getattr(__import__(module, {}, {}, [clsname]), clsname) + encoding = options.get('encoding', None) + + extract_text = options.get('extract_text', True) + if isinstance(extract_text, basestring): + extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true') + + ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS) + if isinstance(ignore_tags, basestring): + ignore_tags = ignore_tags.split() + ignore_tags = [QName(tag) for tag in ignore_tags] + + include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS) + if isinstance(include_attrs, basestring): + include_attrs = include_attrs.split() + include_attrs = [QName(attr) for attr in include_attrs] + + tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None), + encoding=encoding) + translator = Translator(None, ignore_tags, include_attrs, extract_text) + for message in translator.extract(tmpl.stream, gettext_functions=keywords): + yield message
--- a/genshi/filters/tests/__init__.py +++ b/genshi/filters/tests/__init__.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2007 Edgewall Software +# Copyright (C) 2007-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -15,10 +15,12 @@ import unittest def suite(): - from genshi.filters.tests import html, i18n + from genshi.filters.tests import html, i18n, transform suite = unittest.TestSuite() suite.addTest(html.suite()) suite.addTest(i18n.suite()) + if hasattr(doctest, 'NORMALIZE_WHITESPACE'): + suite.addTest(transform.suite()) return suite if __name__ == '__main__':
--- a/genshi/filters/tests/html.py +++ b/genshi/filters/tests/html.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -16,7 +16,7 @@ from genshi.input import HTML, ParseError from genshi.filters.html import HTMLFormFiller, HTMLSanitizer - +from genshi.template import MarkupTemplate class HTMLFormFillerTestCase(unittest.TestCase): @@ -270,6 +270,42 @@ </select> </p></form>""", unicode(html)) + def test_fill_option_segmented_text(self): + html = MarkupTemplate("""<form> + <select name="foo"> + <option value="1">foo $x</option> + </select> + </form>""").generate(x=1) | HTMLFormFiller(data={'foo': '1'}) + self.assertEquals("""<form> + <select name="foo"> + <option value="1" selected="selected">foo 1</option> + </select> + </form>""", unicode(html)) + + def test_fill_option_segmented_text_no_value(self): + html = MarkupTemplate("""<form> + <select name="foo"> + <option>foo $x bar</option> + </select> + </form>""").generate(x=1) | HTMLFormFiller(data={'foo': 'foo 1 bar'}) + self.assertEquals("""<form> + <select name="foo"> + <option selected="selected">foo 1 bar</option> + </select> + </form>""", unicode(html)) + + def test_fill_option_unicode_value(self): + html = HTML(u"""<form> + <select name="foo"> + <option value="ö">foo</option> + </select> + </form>""") | HTMLFormFiller(data={'foo': u'ö'}) + self.assertEquals(u"""<form> + <select name="foo"> + <option value="ö" selected="selected">foo</option> + </select> + </form>""", unicode(html)) + class HTMLSanitizerTestCase(unittest.TestCase): @@ -318,6 +354,10 @@ html = HTML('<div onclick=\'alert("foo")\' />') self.assertEquals(u'<div/>', unicode(html | HTMLSanitizer())) + def test_sanitize_remove_comments(self): + html = HTML('''<div><!-- conditional comment crap --></div>''') + self.assertEquals(u'<div/>', unicode(html | HTMLSanitizer())) + def test_sanitize_remove_style_scripts(self): sanitizer = HTMLSanitizer(safe_attrs=HTMLSanitizer.SAFE_ATTRS | set(['style'])) # Inline style with url() using javascript: scheme @@ -332,6 +372,8 @@ # IE expressions in CSS not allowed html = HTML('<DIV STYLE=\'width: expression(alert("foo"));\'>') self.assertEquals(u'<div/>', unicode(html | sanitizer)) + html = HTML('<DIV STYLE=\'width: e/**/xpression(alert("foo"));\'>') + self.assertEquals(u'<div/>', unicode(html | sanitizer)) html = HTML('<DIV STYLE=\'background: url(javascript:alert("foo"));' 'color: #fff\'>') self.assertEquals(u'<div style="color: #fff"/>',
--- a/genshi/filters/tests/i18n.py +++ b/genshi/filters/tests/i18n.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2007 Edgewall Software +# Copyright (C) 2007-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -11,16 +11,59 @@ # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. +from datetime import datetime import doctest +from gettext import NullTranslations from StringIO import StringIO import unittest +from genshi.core import Attrs from genshi.template import MarkupTemplate -from genshi.filters.i18n import Translator +from genshi.filters.i18n import Translator, extract +from genshi.input import HTML + + +class DummyTranslations(NullTranslations): + + def __init__(self, catalog): + NullTranslations.__init__(self) + self._catalog = catalog + + def ugettext(self, message): + missing = object() + tmsg = self._catalog.get(message, missing) + if tmsg is missing: + if self._fallback: + return self._fallback.ugettext(message) + return unicode(message) + return tmsg class TranslatorTestCase(unittest.TestCase): + def test_translate_included_attribute_text(self): + """ + Verify that translated attributes end up in a proper `Attrs` instance. + """ + html = HTML("""<html> + <span title="Foo"></span> + </html>""") + translator = Translator(lambda s: u"Voh") + stream = list(html.filter(translator)) + kind, data, pos = stream[2] + assert isinstance(data[1], Attrs) + + def test_extract_without_text(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> + <p title="Bar">Foo</p> + ${ngettext("Singular", "Plural", num)} + </html>""") + translator = Translator(extract_text=False) + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual((3, 'ngettext', (u'Singular', u'Plural', None), []), + messages[0]) + def test_extract_plural_form(self): tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> ${ngettext("Singular", "Plural", num)} @@ -28,7 +71,26 @@ translator = Translator() messages = list(translator.extract(tmpl.stream)) self.assertEqual(1, len(messages)) - self.assertEqual((2, 'ngettext', (u'Singular', u'Plural')), messages[0]) + self.assertEqual((2, 'ngettext', (u'Singular', u'Plural', None), []), + messages[0]) + + def test_extract_funky_plural_form(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> + ${ngettext(len(items), *widget.display_names)} + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual((2, 'ngettext', (None, None), []), messages[0]) + + def test_extract_gettext_with_unicode_string(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> + ${gettext("Grüße")} + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual((2, 'gettext', u'Gr\xfc\xdfe', []), messages[0]) def test_extract_included_attribute_text(self): tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> @@ -37,7 +99,7 @@ translator = Translator() messages = list(translator.extract(tmpl.stream)) self.assertEqual(1, len(messages)) - self.assertEqual((2, None, u'Foo'), messages[0]) + self.assertEqual((2, None, u'Foo', []), messages[0]) def test_extract_attribute_expr(self): tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> @@ -46,7 +108,7 @@ translator = Translator() messages = list(translator.extract(tmpl.stream)) self.assertEqual(1, len(messages)) - self.assertEqual((2, '_', u'Save'), messages[0]) + self.assertEqual((2, '_', u'Save', []), messages[0]) def test_extract_non_included_attribute_interpolated(self): tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> @@ -55,7 +117,7 @@ translator = Translator() messages = list(translator.extract(tmpl.stream)) self.assertEqual(1, len(messages)) - self.assertEqual((2, None, u'Foo'), messages[0]) + self.assertEqual((2, None, u'Foo', []), messages[0]) def test_extract_text_from_sub(self): tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> @@ -64,13 +126,449 @@ translator = Translator() messages = list(translator.extract(tmpl.stream)) self.assertEqual(1, len(messages)) - self.assertEqual((2, None, u'Foo'), messages[0]) + self.assertEqual((2, None, u'Foo', []), messages[0]) + + def test_ignore_tag_with_fixed_xml_lang(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> + <p xml:lang="en">(c) 2007 Edgewall Software</p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(0, len(messages)) + + def test_extract_tag_with_variable_xml_lang(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> + <p xml:lang="${lang}">(c) 2007 Edgewall Software</p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual((2, None, u'(c) 2007 Edgewall Software', []), + messages[0]) + + def test_ignore_attribute_with_expression(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> + <input type="submit" value="Reply" title="Reply to comment $num" /> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(0, len(messages)) + + def test_extract_i18n_msg(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Please see <a href="help.html">Help</a> for details. + </p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual('Please see [1:Help] for details.', messages[0][2]) + + def test_translate_i18n_msg(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Please see <a href="help.html">Help</a> for details. + </p> + </html>""") + gettext = lambda s: u"Für Details siehe bitte [1:Hilfe]." + translator = Translator(gettext) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p>Für Details siehe bitte <a href="help.html">Hilfe</a>.</p> + </html>""", tmpl.generate().render()) + + def test_extract_i18n_msg_nested(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Please see <a href="help.html"><em>Help</em> page</a> for details. + </p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual('Please see [1:[2:Help] page] for details.', + messages[0][2]) + + def test_translate_i18n_msg_nested(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Please see <a href="help.html"><em>Help</em> page</a> for details. + </p> + </html>""") + gettext = lambda s: u"Für Details siehe bitte [1:[2:Hilfeseite]]." + translator = Translator(gettext) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p>Für Details siehe bitte <a href="help.html"><em>Hilfeseite</em></a>.</p> + </html>""", tmpl.generate().render()) + + def test_extract_i18n_msg_empty(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Show me <input type="text" name="num" /> entries per page. + </p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual('Show me [1:] entries per page.', messages[0][2]) + + def test_translate_i18n_msg_empty(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Show me <input type="text" name="num" /> entries per page. + </p> + </html>""") + gettext = lambda s: u"[1:] Einträge pro Seite anzeigen." + translator = Translator(gettext) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p><input type="text" name="num"/> Einträge pro Seite anzeigen.</p> + </html>""", tmpl.generate().render()) + + def test_extract_i18n_msg_multiple(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Please see <a href="help.html">Help</a> for <em>details</em>. + </p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual('Please see [1:Help] for [2:details].', messages[0][2]) + + def test_translate_i18n_msg_multiple(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Please see <a href="help.html">Help</a> for <em>details</em>. + </p> + </html>""") + gettext = lambda s: u"Für [2:Details] siehe bitte [1:Hilfe]." + translator = Translator(gettext) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p>Für <em>Details</em> siehe bitte <a href="help.html">Hilfe</a>.</p> + </html>""", tmpl.generate().render()) + + def test_extract_i18n_msg_multiple_empty(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Show me <input type="text" name="num" /> entries per page, starting at page <input type="text" name="num" />. + </p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual('Show me [1:] entries per page, starting at page [2:].', + messages[0][2]) + + def test_translate_i18n_msg_multiple_empty(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Show me <input type="text" name="num" /> entries per page, starting at page <input type="text" name="num" />. + </p> + </html>""") + gettext = lambda s: u"[1:] Einträge pro Seite, beginnend auf Seite [2:]." + translator = Translator(gettext) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p><input type="text" name="num"/> Eintr\xc3\xa4ge pro Seite, beginnend auf Seite <input type="text" name="num"/>.</p> + </html>""", tmpl.generate().render()) + + def test_extract_i18n_msg_with_param(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg="name"> + Hello, ${user.name}! + </p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual('Hello, %(name)s!', messages[0][2]) + + def test_translate_i18n_msg_with_param(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg="name"> + Hello, ${user.name}! + </p> + </html>""") + gettext = lambda s: u"Hallo, %(name)s!" + translator = Translator(gettext) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p>Hallo, Jim!</p> + </html>""", tmpl.generate(user=dict(name='Jim')).render()) + + def test_translate_i18n_msg_with_param_reordered(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg="name"> + Hello, ${user.name}! + </p> + </html>""") + gettext = lambda s: u"%(name)s, sei gegrüßt!" + translator = Translator(gettext) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p>Jim, sei gegrüßt!</p> + </html>""", tmpl.generate(user=dict(name='Jim')).render()) + + def test_translate_i18n_msg_with_attribute_param(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Hello, <a href="#${anchor}">dude</a>! + </p> + </html>""") + gettext = lambda s: u"Sei gegrüßt, [1:Alter]!" + translator = Translator(gettext) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p>Sei gegrüßt, <a href="#42">Alter</a>!</p> + </html>""", tmpl.generate(anchor='42').render()) + + def test_extract_i18n_msg_with_two_params(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg="name, time"> + Posted by ${post.author} at ${entry.time.strftime('%H:%m')} + </p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual('Posted by %(name)s at %(time)s', messages[0][2]) + + def test_translate_i18n_msg_with_two_params(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg="name, time"> + Written by ${entry.author} at ${entry.time.strftime('%H:%M')} + </p> + </html>""") + gettext = lambda s: u"%(name)s schrieb dies um %(time)s" + translator = Translator(gettext) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + entry = { + 'author': 'Jim', + 'time': datetime(2008, 4, 1, 14, 30) + } + self.assertEqual("""<html> + <p>Jim schrieb dies um 14:30</p> + </html>""", tmpl.generate(entry=entry).render()) + + def test_extract_i18n_msg_with_directive(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg=""> + Show me <input type="text" name="num" py:attrs="{'value': x}" /> entries per page. + </p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual('Show me [1:] entries per page.', messages[0][2]) + + # FIXME: this currently fails :-/ +# def test_translate_i18n_msg_with_directive(self): +# tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" +# xmlns:i18n="http://genshi.edgewall.org/i18n"> +# <p i18n:msg=""> +# Show me <input type="text" name="num" py:attrs="{'value': x}" /> entries per page. +# </p> +# </html>""") +# gettext = lambda s: u"[1:] Einträge pro Seite anzeigen." +# tmpl.filters.insert(0, Translator(gettext)) +# self.assertEqual("""<html> +# <p><input type="text" name="num" value="x"/> Einträge pro Seite anzeigen.</p> +# </html>""", tmpl.generate().render()) + + def test_extract_i18n_msg_with_comment(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg="" i18n:comment="As in foo bar">Foo</p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(1, len(messages)) + self.assertEqual((3, None, u'Foo', ['As in foo bar']), messages[0]) + + def test_translate_i18n_msg_with_comment(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg="" i18n:comment="As in foo bar">Foo</p> + </html>""") + gettext = lambda s: u"Voh" + translator = Translator(gettext) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p>Voh</p> + </html>""", tmpl.generate().render()) + + def test_extract_i18n_msg_with_attr(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg="" title="Foo bar">Foo</p> + </html>""") + translator = Translator() + messages = list(translator.extract(tmpl.stream)) + self.assertEqual(2, len(messages)) + self.assertEqual((3, None, u'Foo bar', []), messages[0]) + self.assertEqual((3, None, u'Foo', []), messages[1]) + + def test_translate_i18n_msg_with_attr(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg="" title="Foo bar">Foo</p> + </html>""") + gettext = lambda s: u"Voh" + translator = Translator(DummyTranslations({ + 'Foo': u'Voh', + 'Foo bar': u'Voh bär' + })) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p title="Voh bär">Voh</p> + </html>""", tmpl.generate().render()) + + def test_translate_with_translations_object(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:i18n="http://genshi.edgewall.org/i18n"> + <p i18n:msg="" i18n:comment="As in foo bar">Foo</p> + </html>""") + translator = Translator(DummyTranslations({'Foo': 'Voh'})) + tmpl.filters.insert(0, translator) + tmpl.add_directives(Translator.NAMESPACE, translator) + self.assertEqual("""<html> + <p>Voh</p> + </html>""", tmpl.generate().render()) + + +class ExtractTestCase(unittest.TestCase): + + def test_markup_template_extraction(self): + buf = StringIO("""<html xmlns:py="http://genshi.edgewall.org/"> + <head> + <title>Example</title> + </head> + <body> + <h1>Example</h1> + <p>${_("Hello, %(name)s") % dict(name=username)}</p> + <p>${ngettext("You have %d item", "You have %d items", num)}</p> + </body> + </html>""") + results = list(extract(buf, ['_', 'ngettext'], [], {})) + self.assertEqual([ + (3, None, u'Example', []), + (6, None, u'Example', []), + (7, '_', u'Hello, %(name)s', []), + (8, 'ngettext', (u'You have %d item', u'You have %d items', None), + []), + ], results) + + def test_extraction_without_text(self): + buf = StringIO("""<html xmlns:py="http://genshi.edgewall.org/"> + <p title="Bar">Foo</p> + ${ngettext("Singular", "Plural", num)} + </html>""") + results = list(extract(buf, ['_', 'ngettext'], [], { + 'extract_text': 'no' + })) + self.assertEqual([ + (3, 'ngettext', (u'Singular', u'Plural', None), []), + ], results) + + def test_text_template_extraction(self): + buf = StringIO("""${_("Dear %(name)s") % {'name': name}}, + + ${ngettext("Your item:", "Your items", len(items))} + #for item in items + * $item + #end + + All the best, + Foobar""") + results = list(extract(buf, ['_', 'ngettext'], [], { + 'template_class': 'genshi.template:TextTemplate' + })) + self.assertEqual([ + (1, '_', u'Dear %(name)s', []), + (3, 'ngettext', (u'Your item:', u'Your items', None), []), + (7, None, u'All the best,\n Foobar', []) + ], results) + + def test_extraction_with_keyword_arg(self): + buf = StringIO("""<html xmlns:py="http://genshi.edgewall.org/"> + ${gettext('Foobar', foo='bar')} + </html>""") + results = list(extract(buf, ['gettext'], [], {})) + self.assertEqual([ + (2, 'gettext', (u'Foobar'), []), + ], results) + + def test_extraction_with_nonstring_arg(self): + buf = StringIO("""<html xmlns:py="http://genshi.edgewall.org/"> + ${dgettext(curdomain, 'Foobar')} + </html>""") + results = list(extract(buf, ['dgettext'], [], {})) + self.assertEqual([ + (2, 'dgettext', (None, u'Foobar'), []), + ], results) + + def test_extraction_inside_ignored_tags(self): + buf = StringIO("""<html xmlns:py="http://genshi.edgewall.org/"> + <script type="text/javascript"> + $('#llist').tabs({ + remote: true, + spinner: "${_('Please wait...')}" + }); + </script> + </html>""") + results = list(extract(buf, ['_'], [], {})) + self.assertEqual([ + (5, '_', u'Please wait...', []), + ], results) + + def test_extraction_inside_ignored_tags_with_directives(self): + buf = StringIO("""<html xmlns:py="http://genshi.edgewall.org/"> + <script type="text/javascript"> + <py:if test="foobar"> + alert("This shouldn't be extracted"); + </py:if> + </script> + </html>""") + self.assertEqual([], list(extract(buf, ['_'], [], {}))) def suite(): suite = unittest.TestSuite() + suite.addTest(doctest.DocTestSuite(Translator.__module__)) suite.addTest(unittest.makeSuite(TranslatorTestCase, 'test')) - suite.addTests(doctest.DocTestSuite(Translator.__module__)) + suite.addTest(unittest.makeSuite(ExtractTestCase, 'test')) return suite if __name__ == '__main__':
new file mode 100644 --- /dev/null +++ b/genshi/filters/tests/transform.py @@ -0,0 +1,1497 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2007 Edgewall Software +# All rights reserved. +# +# This software is licensed as described in the file COPYING, which +# you should have received as part of this distribution. The terms +# are also available at http://genshi.edgewall.org/wiki/License. +# +# This software consists of voluntary contributions made by many +# individuals. For the exact contribution history, see the revision +# history and logs, available at http://genshi.edgewall.org/log/. + +import doctest +from pprint import pprint +import unittest + +from genshi import HTML +from genshi.builder import Element +from genshi.core import START, END, TEXT, QName, Attrs +from genshi.filters.transform import Transformer, StreamBuffer, ENTER, EXIT, \ + OUTSIDE, INSIDE, ATTR, BREAK +import genshi.filters.transform + + +FOO = '<root>ROOT<foo name="foo">FOO</foo></root>' +FOOBAR = '<root>ROOT<foo name="foo" size="100">FOO</foo><bar name="bar">BAR</bar></root>' + + +def _simplify(stream, with_attrs=False): + """Simplify a marked stream.""" + def _generate(): + for mark, (kind, data, pos) in stream: + if kind is START: + if with_attrs: + data = (unicode(data[0]), dict((unicode(k), v) + for k, v in data[1])) + else: + data = unicode(data[0]) + elif kind is END: + data = unicode(data) + elif kind is ATTR: + kind = ATTR + data = dict((unicode(k), v) for k, v in data[1]) + yield mark, kind, data + return list(_generate()) + + +def _transform(html, transformer, with_attrs=False): + """Apply transformation returning simplified marked stream.""" + if isinstance(html, basestring): + html = HTML(html) + stream = transformer(html, keep_marks=True) + return _simplify(stream, with_attrs) + + +class SelectTest(unittest.TestCase): + """Test .select()""" + def _select(self, select): + html = HTML(FOOBAR) + if isinstance(select, basestring): + select = [select] + transformer = Transformer(select[0]) + for sel in select[1:]: + transformer = transformer.select(sel) + return _transform(html, transformer) + + def test_select_single_element(self): + self.assertEqual( + self._select('foo'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')], + ) + + def test_select_context(self): + self.assertEqual( + self._select('.'), + [(ENTER, START, u'root'), + (INSIDE, TEXT, u'ROOT'), + (INSIDE, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (INSIDE, END, u'foo'), + (INSIDE, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (INSIDE, END, u'bar'), + (EXIT, END, u'root')] + ) + + def test_select_inside_select(self): + self.assertEqual( + self._select(['.', 'foo']), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')], + ) + + def test_select_text(self): + self.assertEqual( + self._select('*/text()'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'foo'), + (OUTSIDE, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, u'bar'), + (OUTSIDE, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')], + ) + + def test_select_attr(self): + self.assertEqual( + self._select('foo/@name'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ATTR, ATTR, {'name': u'foo'}), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_select_text_context(self): + self.assertEqual( + list(Transformer('.')(HTML('foo'), keep_marks=True)), + [('OUTSIDE', ('TEXT', u'foo', (None, 1, 0)))], + ) + + +class InvertTest(unittest.TestCase): + def _invert(self, select): + return _transform(FOO, Transformer(select).invert()) + + def test_invert_element(self): + self.assertEqual( + self._invert('foo'), + [(OUTSIDE, START, u'root'), + (OUTSIDE, TEXT, u'ROOT'), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (OUTSIDE, END, u'root')] + ) + + def test_invert_inverted_element(self): + self.assertEqual( + _transform(FOO, Transformer('foo').invert().invert()), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (OUTSIDE, START, u'foo'), + (OUTSIDE, TEXT, u'FOO'), + (OUTSIDE, END, u'foo'), + (None, END, u'root')] + ) + + def test_invert_text(self): + self.assertEqual( + self._invert('foo/text()'), + [(OUTSIDE, START, u'root'), + (OUTSIDE, TEXT, u'ROOT'), + (OUTSIDE, START, u'foo'), + (None, TEXT, u'FOO'), + (OUTSIDE, END, u'foo'), + (OUTSIDE, END, u'root')] + ) + + def test_invert_attribute(self): + self.assertEqual( + self._invert('foo/@name'), + [(OUTSIDE, START, u'root'), + (OUTSIDE, TEXT, u'ROOT'), + (None, ATTR, {'name': u'foo'}), + (OUTSIDE, START, u'foo'), + (OUTSIDE, TEXT, u'FOO'), + (OUTSIDE, END, u'foo'), + (OUTSIDE, END, u'root')] + ) + + def test_invert_context(self): + self.assertEqual( + self._invert('.'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, END, u'root')] + ) + + def test_invert_text_context(self): + self.assertEqual( + _simplify(Transformer('.').invert()(HTML('foo'), keep_marks=True)), + [(None, 'TEXT', u'foo')], + ) + + + +class EndTest(unittest.TestCase): + def test_end(self): + stream = _transform(FOO, Transformer('foo').end()) + self.assertEqual( + stream, + [(OUTSIDE, START, u'root'), + (OUTSIDE, TEXT, u'ROOT'), + (OUTSIDE, START, u'foo'), + (OUTSIDE, TEXT, u'FOO'), + (OUTSIDE, END, u'foo'), + (OUTSIDE, END, u'root')] + ) + + +class EmptyTest(unittest.TestCase): + def _empty(self, select): + return _transform(FOO, Transformer(select).empty()) + + def test_empty_element(self): + self.assertEqual( + self._empty('foo'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (EXIT, END, u'foo'), + (None, END, u'root')], + ) + + def test_empty_text(self): + self.assertEqual( + self._empty('foo/text()'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'foo'), + (OUTSIDE, TEXT, u'FOO'), + (None, END, u'foo'), + (None, END, u'root')] + ) + + def test_empty_attr(self): + self.assertEqual( + self._empty('foo/@name'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ATTR, ATTR, {'name': u'foo'}), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, END, u'root')] + ) + + def test_empty_context(self): + self.assertEqual( + self._empty('.'), + [(ENTER, START, u'root'), + (EXIT, END, u'root')] + ) + + def test_empty_text_context(self): + self.assertEqual( + _simplify(Transformer('.')(HTML('foo'), keep_marks=True)), + [(OUTSIDE, TEXT, u'foo')], + ) + + +class RemoveTest(unittest.TestCase): + def _remove(self, select): + return _transform(FOO, Transformer(select).remove()) + + def test_remove_element(self): + self.assertEqual( + self._remove('foo|bar'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, END, u'root')] + ) + + def test_remove_text(self): + self.assertEqual( + self._remove('//text()'), + [(None, START, u'root'), + (None, START, u'foo'), + (None, END, u'foo'), + (None, END, u'root')] + ) + + def test_remove_attr(self): + self.assertEqual( + self._remove('foo/@name'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, END, u'root')] + ) + + def test_remove_context(self): + self.assertEqual( + self._remove('.'), + [], + ) + + def test_remove_text_context(self): + self.assertEqual( + _transform('foo', Transformer('.').remove()), + [], + ) + + +class UnwrapText(unittest.TestCase): + def _unwrap(self, select): + return _transform(FOO, Transformer(select).unwrap()) + + def test_unwrap_element(self): + self.assertEqual( + self._unwrap('foo'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (INSIDE, TEXT, u'FOO'), + (None, END, u'root')] + ) + + def test_unwrap_text(self): + self.assertEqual( + self._unwrap('foo/text()'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'foo'), + (OUTSIDE, TEXT, u'FOO'), + (None, END, u'foo'), + (None, END, u'root')] + ) + + def test_unwrap_attr(self): + self.assertEqual( + self._unwrap('foo/@name'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ATTR, ATTR, {'name': u'foo'}), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, END, u'root')] + ) + + def test_unwrap_adjacent(self): + self.assertEqual( + _transform(FOOBAR, Transformer('foo|bar').unwrap()), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (INSIDE, TEXT, u'FOO'), + (INSIDE, TEXT, u'BAR'), + (None, END, u'root')] + ) + + def test_unwrap_root(self): + self.assertEqual( + self._unwrap('.'), + [(INSIDE, TEXT, u'ROOT'), + (INSIDE, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (INSIDE, END, u'foo')] + ) + + def test_unwrap_text_root(self): + self.assertEqual( + _transform('foo', Transformer('.').unwrap()), + [(OUTSIDE, TEXT, 'foo')], + ) + + +class WrapTest(unittest.TestCase): + def _wrap(self, select, wrap='wrap'): + return _transform(FOO, Transformer(select).wrap(wrap)) + + def test_wrap_element(self): + self.assertEqual( + self._wrap('foo'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'wrap'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, END, u'wrap'), + (None, END, u'root')] + ) + + def test_wrap_adjacent_elements(self): + self.assertEqual( + _transform(FOOBAR, Transformer('foo|bar').wrap('wrap')), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'wrap'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, END, u'wrap'), + (None, START, u'wrap'), + (ENTER, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (EXIT, END, u'bar'), + (None, END, u'wrap'), + (None, END, u'root')] + ) + + def test_wrap_text(self): + self.assertEqual( + self._wrap('foo/text()'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'foo'), + (None, START, u'wrap'), + (OUTSIDE, TEXT, u'FOO'), + (None, END, u'wrap'), + (None, END, u'foo'), + (None, END, u'root')] + ) + + def test_wrap_root(self): + self.assertEqual( + self._wrap('.'), + [(None, START, u'wrap'), + (ENTER, START, u'root'), + (INSIDE, TEXT, u'ROOT'), + (INSIDE, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (INSIDE, END, u'foo'), + (EXIT, END, u'root'), + (None, END, u'wrap')] + ) + + def test_wrap_text_root(self): + self.assertEqual( + _transform('foo', Transformer('.').wrap('wrap')), + [(None, START, u'wrap'), + (OUTSIDE, TEXT, u'foo'), + (None, END, u'wrap')], + ) + + def test_wrap_with_element(self): + element = Element('a', href='http://localhost') + self.assertEqual( + _transform('foo', Transformer('.').wrap(element), with_attrs=True), + [(None, START, (u'a', {u'href': u'http://localhost'})), + (OUTSIDE, TEXT, u'foo'), + (None, END, u'a')] + ) + + +class FilterTest(unittest.TestCase): + def _filter(self, select, html=FOOBAR): + """Returns a list of lists of filtered elements.""" + output = [] + def filtered(stream): + interval = [] + output.append(interval) + for event in stream: + interval.append(event) + yield event + _transform(html, Transformer(select).filter(filtered)) + simplified = [] + for sub in output: + simplified.append(_simplify([(None, event) for event in sub])) + return simplified + + def test_filter_element(self): + self.assertEqual( + self._filter('foo'), + [[(None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo')]] + ) + + def test_filter_adjacent_elements(self): + self.assertEqual( + self._filter('foo|bar'), + [[(None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo')], + [(None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar')]] + ) + + def test_filter_text(self): + self.assertEqual( + self._filter('*/text()'), + [[(None, TEXT, u'FOO')], + [(None, TEXT, u'BAR')]] + ) + def test_filter_root(self): + self.assertEqual( + self._filter('.'), + [[(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')]] + ) + + def test_filter_text_root(self): + self.assertEqual( + self._filter('.', 'foo'), + [[(None, TEXT, u'foo')]]) + + def test_filter_after_outside(self): + stream = _transform( + '<root>x</root>', Transformer('//root/text()').filter(lambda x: x)) + self.assertEqual( + list(stream), + [(None, START, u'root'), + (OUTSIDE, TEXT, u'x'), + (None, END, u'root')]) + + +class MapTest(unittest.TestCase): + def _map(self, select, kind=None): + data = [] + def record(d): + data.append(d) + return d + _transform(FOOBAR, Transformer(select).map(record, kind)) + return data + + def test_map_element(self): + self.assertEqual( + self._map('foo'), + [(QName(u'foo'), Attrs([(QName(u'name'), u'foo'), + (QName(u'size'), u'100')])), + u'FOO', + QName(u'foo')] + ) + + def test_map_with_text_kind(self): + self.assertEqual( + self._map('.', TEXT), + [u'ROOT', u'FOO', u'BAR'] + ) + + def test_map_with_root_and_end_kind(self): + self.assertEqual( + self._map('.', END), + [QName(u'foo'), QName(u'bar'), QName(u'root')] + ) + + def test_map_with_attribute(self): + self.assertEqual( + self._map('foo/@name'), + [(QName(u'foo@*'), Attrs([('name', u'foo')]))] + ) + + +class SubstituteTest(unittest.TestCase): + def _substitute(self, select, pattern, replace): + return _transform(FOOBAR, Transformer(select).substitute(pattern, replace)) + + def test_substitute_foo(self): + self.assertEqual( + self._substitute('foo', 'FOO|BAR', 'FOOOOO'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOOOOO'), + (EXIT, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_substitute_foobar_with_group(self): + self.assertEqual( + self._substitute('foo|bar', '(FOO|BAR)', r'(\1)'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'(FOO)'), + (EXIT, END, u'foo'), + (ENTER, START, u'bar'), + (INSIDE, TEXT, u'(BAR)'), + (EXIT, END, u'bar'), + (None, END, u'root')] + ) + + +class RenameTest(unittest.TestCase): + def _rename(self, select): + return _transform(FOOBAR, Transformer(select).rename('foobar')) + + def test_rename_root(self): + self.assertEqual( + self._rename('.'), + [(ENTER, START, u'foobar'), + (INSIDE, TEXT, u'ROOT'), + (INSIDE, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (INSIDE, END, u'foo'), + (INSIDE, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (INSIDE, END, u'bar'), + (EXIT, END, u'foobar')] + ) + + def test_rename_element(self): + self.assertEqual( + self._rename('foo|bar'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foobar'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foobar'), + (ENTER, START, u'foobar'), + (INSIDE, TEXT, u'BAR'), + (EXIT, END, u'foobar'), + (None, END, u'root')] + ) + + def test_rename_text(self): + self.assertEqual( + self._rename('foo/text()'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'foo'), + (OUTSIDE, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + +class ContentTestMixin(object): + def _apply(self, select, content=None, html=FOOBAR): + class Injector(object): + count = 0 + + def __iter__(self): + self.count += 1 + return iter(HTML('CONTENT %i' % self.count)) + + if isinstance(html, basestring): + html = HTML(html) + if content is None: + content = Injector() + elif isinstance(content, basestring): + content = HTML(content) + return _transform(html, getattr(Transformer(select), self.operation) + (content)) + + +class ReplaceTest(unittest.TestCase, ContentTestMixin): + operation = 'replace' + + def test_replace_element(self): + self.assertEqual( + self._apply('foo'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, TEXT, u'CONTENT 1'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_replace_text(self): + self.assertEqual( + self._apply('text()'), + [(None, START, u'root'), + (None, TEXT, u'CONTENT 1'), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_replace_context(self): + self.assertEqual( + self._apply('.'), + [(None, TEXT, u'CONTENT 1')], + ) + + def test_replace_text_context(self): + self.assertEqual( + self._apply('.', html='foo'), + [(None, TEXT, u'CONTENT 1')], + ) + + def test_replace_adjacent_elements(self): + self.assertEqual( + self._apply('*'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, TEXT, u'CONTENT 1'), + (None, TEXT, u'CONTENT 2'), + (None, END, u'root')], + ) + + def test_replace_all(self): + self.assertEqual( + self._apply('*|text()'), + [(None, START, u'root'), + (None, TEXT, u'CONTENT 1'), + (None, TEXT, u'CONTENT 2'), + (None, TEXT, u'CONTENT 3'), + (None, END, u'root')], + ) + + def test_replace_with_callback(self): + count = [0] + def content(): + count[0] += 1 + yield '%2i.' % count[0] + self.assertEqual( + self._apply('*', content), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, TEXT, u' 1.'), + (None, TEXT, u' 2.'), + (None, END, u'root')] + ) + + +class BeforeTest(unittest.TestCase, ContentTestMixin): + operation = 'before' + + def test_before_element(self): + self.assertEqual( + self._apply('foo'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, TEXT, u'CONTENT 1'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_before_text(self): + self.assertEqual( + self._apply('text()'), + [(None, START, u'root'), + (None, TEXT, u'CONTENT 1'), + (OUTSIDE, TEXT, u'ROOT'), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_before_context(self): + self.assertEqual( + self._apply('.'), + [(None, TEXT, u'CONTENT 1'), + (ENTER, START, u'root'), + (INSIDE, TEXT, u'ROOT'), + (INSIDE, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (INSIDE, END, u'foo'), + (INSIDE, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (INSIDE, END, u'bar'), + (EXIT, END, u'root')] + ) + + def test_before_text_context(self): + self.assertEqual( + self._apply('.', html='foo'), + [(None, TEXT, u'CONTENT 1'), + (OUTSIDE, TEXT, u'foo')] + ) + + def test_before_adjacent_elements(self): + self.assertEqual( + self._apply('*'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, TEXT, u'CONTENT 1'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, TEXT, u'CONTENT 2'), + (ENTER, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (EXIT, END, u'bar'), + (None, END, u'root')] + + ) + + def test_before_all(self): + self.assertEqual( + self._apply('*|text()'), + [(None, START, u'root'), + (None, TEXT, u'CONTENT 1'), + (OUTSIDE, TEXT, u'ROOT'), + (None, TEXT, u'CONTENT 2'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, TEXT, u'CONTENT 3'), + (ENTER, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (EXIT, END, u'bar'), + (None, END, u'root')] + ) + + def test_before_with_callback(self): + count = [0] + def content(): + count[0] += 1 + yield '%2i.' % count[0] + self.assertEqual( + self._apply('foo/text()', content), + [(None, 'START', u'root'), + (None, 'TEXT', u'ROOT'), + (None, 'START', u'foo'), + (None, 'TEXT', u' 1.'), + ('OUTSIDE', 'TEXT', u'FOO'), + (None, 'END', u'foo'), + (None, 'START', u'bar'), + (None, 'TEXT', u'BAR'), + (None, 'END', u'bar'), + (None, 'END', u'root')] + ) + + +class AfterTest(unittest.TestCase, ContentTestMixin): + operation = 'after' + + def test_after_element(self): + self.assertEqual( + self._apply('foo'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, TEXT, u'CONTENT 1'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_after_text(self): + self.assertEqual( + self._apply('text()'), + [(None, START, u'root'), + (OUTSIDE, TEXT, u'ROOT'), + (None, TEXT, u'CONTENT 1'), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_after_context(self): + self.assertEqual( + self._apply('.'), + [(ENTER, START, u'root'), + (INSIDE, TEXT, u'ROOT'), + (INSIDE, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (INSIDE, END, u'foo'), + (INSIDE, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (INSIDE, END, u'bar'), + (EXIT, END, u'root'), + (None, TEXT, u'CONTENT 1')] + ) + + def test_after_text_context(self): + self.assertEqual( + self._apply('.', html='foo'), + [(OUTSIDE, TEXT, u'foo'), + (None, TEXT, u'CONTENT 1')] + ) + + def test_after_adjacent_elements(self): + self.assertEqual( + self._apply('*'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, TEXT, u'CONTENT 1'), + (ENTER, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (EXIT, END, u'bar'), + (None, TEXT, u'CONTENT 2'), + (None, END, u'root')] + + ) + + def test_after_all(self): + self.assertEqual( + self._apply('*|text()'), + [(None, START, u'root'), + (OUTSIDE, TEXT, u'ROOT'), + (None, TEXT, u'CONTENT 1'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, TEXT, u'CONTENT 2'), + (ENTER, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (EXIT, END, u'bar'), + (None, TEXT, u'CONTENT 3'), + (None, END, u'root')] + ) + + def test_after_with_callback(self): + count = [0] + def content(): + count[0] += 1 + yield '%2i.' % count[0] + self.assertEqual( + self._apply('foo/text()', content), + [(None, 'START', u'root'), + (None, 'TEXT', u'ROOT'), + (None, 'START', u'foo'), + ('OUTSIDE', 'TEXT', u'FOO'), + (None, 'TEXT', u' 1.'), + (None, 'END', u'foo'), + (None, 'START', u'bar'), + (None, 'TEXT', u'BAR'), + (None, 'END', u'bar'), + (None, 'END', u'root')] + ) + + +class PrependTest(unittest.TestCase, ContentTestMixin): + operation = 'prepend' + + def test_prepend_element(self): + self.assertEqual( + self._apply('foo'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (None, TEXT, u'CONTENT 1'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_prepend_text(self): + self.assertEqual( + self._apply('text()'), + [(None, START, u'root'), + (OUTSIDE, TEXT, u'ROOT'), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_prepend_context(self): + self.assertEqual( + self._apply('.'), + [(ENTER, START, u'root'), + (None, TEXT, u'CONTENT 1'), + (INSIDE, TEXT, u'ROOT'), + (INSIDE, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (INSIDE, END, u'foo'), + (INSIDE, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (INSIDE, END, u'bar'), + (EXIT, END, u'root')], + ) + + def test_prepend_text_context(self): + self.assertEqual( + self._apply('.', html='foo'), + [(OUTSIDE, TEXT, u'foo')] + ) + + def test_prepend_adjacent_elements(self): + self.assertEqual( + self._apply('*'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (None, TEXT, u'CONTENT 1'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (ENTER, START, u'bar'), + (None, TEXT, u'CONTENT 2'), + (INSIDE, TEXT, u'BAR'), + (EXIT, END, u'bar'), + (None, END, u'root')] + + ) + + def test_prepend_all(self): + self.assertEqual( + self._apply('*|text()'), + [(None, START, u'root'), + (OUTSIDE, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (None, TEXT, u'CONTENT 1'), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (ENTER, START, u'bar'), + (None, TEXT, u'CONTENT 2'), + (INSIDE, TEXT, u'BAR'), + (EXIT, END, u'bar'), + (None, END, u'root')] + ) + + def test_prepend_with_callback(self): + count = [0] + def content(): + count[0] += 1 + yield '%2i.' % count[0] + self.assertEqual( + self._apply('foo', content), + [(None, 'START', u'root'), + (None, 'TEXT', u'ROOT'), + (ENTER, 'START', u'foo'), + (None, 'TEXT', u' 1.'), + (INSIDE, 'TEXT', u'FOO'), + (EXIT, 'END', u'foo'), + (None, 'START', u'bar'), + (None, 'TEXT', u'BAR'), + (None, 'END', u'bar'), + (None, 'END', u'root')] + ) + + +class AppendTest(unittest.TestCase, ContentTestMixin): + operation = 'append' + + def test_append_element(self): + self.assertEqual( + self._apply('foo'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (None, TEXT, u'CONTENT 1'), + (EXIT, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_append_text(self): + self.assertEqual( + self._apply('text()'), + [(None, START, u'root'), + (OUTSIDE, TEXT, u'ROOT'), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_append_context(self): + self.assertEqual( + self._apply('.'), + [(ENTER, START, u'root'), + (INSIDE, TEXT, u'ROOT'), + (INSIDE, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (INSIDE, END, u'foo'), + (INSIDE, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (INSIDE, END, u'bar'), + (None, TEXT, u'CONTENT 1'), + (EXIT, END, u'root')], + ) + + def test_append_text_context(self): + self.assertEqual( + self._apply('.', html='foo'), + [(OUTSIDE, TEXT, u'foo')] + ) + + def test_append_adjacent_elements(self): + self.assertEqual( + self._apply('*'), + [(None, START, u'root'), + (None, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (None, TEXT, u'CONTENT 1'), + (EXIT, END, u'foo'), + (ENTER, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (None, TEXT, u'CONTENT 2'), + (EXIT, END, u'bar'), + (None, END, u'root')] + + ) + + def test_append_all(self): + self.assertEqual( + self._apply('*|text()'), + [(None, START, u'root'), + (OUTSIDE, TEXT, u'ROOT'), + (ENTER, START, u'foo'), + (INSIDE, TEXT, u'FOO'), + (None, TEXT, u'CONTENT 1'), + (EXIT, END, u'foo'), + (ENTER, START, u'bar'), + (INSIDE, TEXT, u'BAR'), + (None, TEXT, u'CONTENT 2'), + (EXIT, END, u'bar'), + (None, END, u'root')] + ) + + def test_append_with_callback(self): + count = [0] + def content(): + count[0] += 1 + yield '%2i.' % count[0] + self.assertEqual( + self._apply('foo', content), + [(None, 'START', u'root'), + (None, 'TEXT', u'ROOT'), + (ENTER, 'START', u'foo'), + (INSIDE, 'TEXT', u'FOO'), + (None, 'TEXT', u' 1.'), + (EXIT, 'END', u'foo'), + (None, 'START', u'bar'), + (None, 'TEXT', u'BAR'), + (None, 'END', u'bar'), + (None, 'END', u'root')] + ) + + + +class AttrTest(unittest.TestCase): + def _attr(self, select, name, value): + return _transform(FOOBAR, Transformer(select).attr(name, value), + with_attrs=True) + + def test_set_existing_attr(self): + self.assertEqual( + self._attr('foo', 'name', 'FOO'), + [(None, START, (u'root', {})), + (None, TEXT, u'ROOT'), + (ENTER, START, (u'foo', {u'name': 'FOO', u'size': '100'})), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, START, (u'bar', {u'name': u'bar'})), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_set_new_attr(self): + self.assertEqual( + self._attr('foo', 'title', 'FOO'), + [(None, START, (u'root', {})), + (None, TEXT, u'ROOT'), + (ENTER, START, (u'foo', {u'name': u'foo', u'title': 'FOO', u'size': '100'})), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, START, (u'bar', {u'name': u'bar'})), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_attr_from_function(self): + def set(name, event): + self.assertEqual(name, 'name') + return event[1][1].get('name').upper() + + self.assertEqual( + self._attr('foo|bar', 'name', set), + [(None, START, (u'root', {})), + (None, TEXT, u'ROOT'), + (ENTER, START, (u'foo', {u'name': 'FOO', u'size': '100'})), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (ENTER, START, (u'bar', {u'name': 'BAR'})), + (INSIDE, TEXT, u'BAR'), + (EXIT, END, u'bar'), + (None, END, u'root')] + ) + + def test_remove_attr(self): + self.assertEqual( + self._attr('foo', 'name', None), + [(None, START, (u'root', {})), + (None, TEXT, u'ROOT'), + (ENTER, START, (u'foo', {u'size': '100'})), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, START, (u'bar', {u'name': u'bar'})), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + def test_remove_attr_with_function(self): + def set(name, event): + return None + + self.assertEqual( + self._attr('foo', 'name', set), + [(None, START, (u'root', {})), + (None, TEXT, u'ROOT'), + (ENTER, START, (u'foo', {u'size': '100'})), + (INSIDE, TEXT, u'FOO'), + (EXIT, END, u'foo'), + (None, START, (u'bar', {u'name': u'bar'})), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')] + ) + + +class BufferTestMixin(object): + def _apply(self, select, with_attrs=False): + buffer = StreamBuffer() + events = buffer.events + + class Trace(object): + last = None + trace = [] + + def __call__(self, stream): + for event in stream: + if events and hash(tuple(events)) != self.last: + self.last = hash(tuple(events)) + self.trace.append(list(events)) + yield event + + trace = Trace() + output = _transform(FOOBAR, getattr(Transformer(select), self.operation) + (buffer).apply(trace), with_attrs=with_attrs) + simplified = [] + for interval in trace.trace: + simplified.append(_simplify([(None, e) for e in interval], + with_attrs=with_attrs)) + return output, simplified + + +class CopyTest(unittest.TestCase, BufferTestMixin): + operation = 'copy' + + def test_copy_element(self): + self.assertEqual( + self._apply('foo')[1], + [[(None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo')]] + ) + + def test_copy_adjacent_elements(self): + self.assertEqual( + self._apply('foo|bar')[1], + [[(None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo')], + [(None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar')]] + ) + + def test_copy_all(self): + self.assertEqual( + self._apply('*|text()')[1], + [[(None, TEXT, u'ROOT')], + [(None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo')], + [(None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar')]] + ) + + def test_copy_text(self): + self.assertEqual( + self._apply('*/text()')[1], + [[(None, TEXT, u'FOO')], + [(None, TEXT, u'BAR')]] + ) + + def test_copy_context(self): + self.assertEqual( + self._apply('.')[1], + [[(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')]] + ) + + def test_copy_attribute(self): + self.assertEqual( + self._apply('foo/@name', with_attrs=True)[1], + [[(None, ATTR, {'name': u'foo'})]] + ) + + def test_copy_attributes(self): + self.assertEqual( + self._apply('foo/@*', with_attrs=True)[1], + [[(None, ATTR, {u'name': u'foo', u'size': u'100'})]] + ) + + +class CutTest(unittest.TestCase, BufferTestMixin): + operation = 'cut' + + def test_cut_element(self): + self.assertEqual( + self._apply('foo'), + ([(None, START, u'root'), + (None, TEXT, u'ROOT'), + (None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')], + [[(None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo')]]) + ) + + def test_cut_adjacent_elements(self): + self.assertEqual( + self._apply('foo|bar'), + ([(None, START, u'root'), + (None, TEXT, u'ROOT'), + (BREAK, BREAK, None), + (None, END, u'root')], + [[(None, START, u'foo'), + (None, TEXT, u'FOO'), + (None, END, u'foo')], + [(None, START, u'bar'), + (None, TEXT, u'BAR'), + (None, END, u'bar')]]) + ) + + def test_cut_all(self): + self.assertEqual( + self._apply('*|text()'), + ([(None, 'START', u'root'), + ('BREAK', 'BREAK', None), + ('BREAK', 'BREAK', None), + (None, 'END', u'root')], + [[(None, 'TEXT', u'ROOT')], + [(None, 'START', u'foo'), + (None, 'TEXT', u'FOO'), + (None, 'END', u'foo')], + [(None, 'START', u'bar'), + (None, 'TEXT', u'BAR'), + (None, 'END', u'bar')]]) + ) + + def test_cut_text(self): + self.assertEqual( + self._apply('*/text()'), + ([(None, 'START', u'root'), + (None, 'TEXT', u'ROOT'), + (None, 'START', u'foo'), + (None, 'END', u'foo'), + (None, 'START', u'bar'), + (None, 'END', u'bar'), + (None, 'END', u'root')], + [[(None, 'TEXT', u'FOO')], + [(None, 'TEXT', u'BAR')]]) + ) + + def test_cut_context(self): + self.assertEqual( + self._apply('.')[1], + [[(None, 'START', u'root'), + (None, 'TEXT', u'ROOT'), + (None, 'START', u'foo'), + (None, 'TEXT', u'FOO'), + (None, 'END', u'foo'), + (None, 'START', u'bar'), + (None, 'TEXT', u'BAR'), + (None, 'END', u'bar'), + (None, 'END', u'root')]] + ) + + def test_cut_attribute(self): + self.assertEqual( + self._apply('foo/@name', with_attrs=True), + ([(None, START, (u'root', {})), + (None, TEXT, u'ROOT'), + (None, START, (u'foo', {u'size': u'100'})), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, (u'bar', {u'name': u'bar'})), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')], + [[(None, ATTR, {u'name': u'foo'})]]) + ) + + def test_cut_attributes(self): + self.assertEqual( + self._apply('foo/@*', with_attrs=True), + ([(None, START, (u'root', {})), + (None, TEXT, u'ROOT'), + (None, START, (u'foo', {})), + (None, TEXT, u'FOO'), + (None, END, u'foo'), + (None, START, (u'bar', {u'name': u'bar'})), + (None, TEXT, u'BAR'), + (None, END, u'bar'), + (None, END, u'root')], + [[(None, ATTR, {u'name': u'foo', u'size': u'100'})]]) + ) + +# XXX Test this when the XPath implementation is fixed (#233). +# def test_cut_attribute_or_attribute(self): +# self.assertEqual( +# self._apply('foo/@name | foo/@size', with_attrs=True), +# ([(None, START, (u'root', {})), +# (None, TEXT, u'ROOT'), +# (None, START, (u'foo', {})), +# (None, TEXT, u'FOO'), +# (None, END, u'foo'), +# (None, START, (u'bar', {u'name': u'bar'})), +# (None, TEXT, u'BAR'), +# (None, END, u'bar'), +# (None, END, u'root')], +# [[(None, ATTR, {u'name': u'foo', u'size': u'100'})]]) +# ) + + + + +def suite(): + from genshi.input import HTML + from genshi.core import Markup + from genshi.builder import tag + suite = unittest.TestSuite() + for test in (SelectTest, InvertTest, EndTest, + EmptyTest, RemoveTest, UnwrapText, WrapTest, FilterTest, + MapTest, SubstituteTest, RenameTest, ReplaceTest, BeforeTest, + AfterTest, PrependTest, AppendTest, AttrTest, CopyTest, CutTest): + suite.addTest(unittest.makeSuite(test, 'test')) + suite.addTest(doctest.DocTestSuite( + genshi.filters.transform, optionflags=doctest.NORMALIZE_WHITESPACE, + extraglobs={'HTML': HTML, 'tag': tag, 'Markup': Markup})) + return suite + + +if __name__ == '__main__': + unittest.main(defaultTest='suite')
new file mode 100644 --- /dev/null +++ b/genshi/filters/transform.py @@ -0,0 +1,1310 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2007 Edgewall Software +# All rights reserved. +# +# This software is licensed as described in the file COPYING, which +# you should have received as part of this distribution. The terms +# are also available at http://genshi.edgewall.org/wiki/License. +# +# This software consists of voluntary contributions made by many +# individuals. For the exact contribution history, see the revision +# history and logs, available at http://genshi.edgewall.org/log/. + +"""A filter for functional-style transformations of markup streams. + +The `Transformer` filter provides a variety of transformations that can be +applied to parts of streams that match given XPath expressions. These +transformations can be chained to achieve results that would be comparitively +tedious to achieve by writing stream filters by hand. The approach of chaining +node selection and transformation has been inspired by the `jQuery`_ Javascript +library. + + .. _`jQuery`: http://jquery.com/ + +For example, the following transformation removes the ``<title>`` element from +the ``<head>`` of the input document: + +>>> from genshi.builder import tag +>>> html = HTML('''<html> +... <head><title>Some Title</title></head> +... <body> +... Some <em>body</em> text. +... </body> +... </html>''') +>>> print html | Transformer('body/em').map(unicode.upper, TEXT) \\ +... .unwrap().wrap(tag.u) +<html> + <head><title>Some Title</title></head> + <body> + Some <u>BODY</u> text. + </body> +</html> + +The ``Transformer`` support a large number of useful transformations out of the +box, but custom transformations can be added easily. + +:since: version 0.5 +""" + +import re +import sys + +from genshi.builder import Element +from genshi.core import Stream, Attrs, QName, TEXT, START, END, _ensure, Markup +from genshi.path import Path + +__all__ = ['Transformer', 'StreamBuffer', 'InjectorTransformation', 'ENTER', + 'EXIT', 'INSIDE', 'OUTSIDE', 'BREAK'] + + +class TransformMark(str): + """A mark on a transformation stream.""" + __slots__ = [] + _instances = {} + + def __new__(cls, val): + return cls._instances.setdefault(val, str.__new__(cls, val)) + + +ENTER = TransformMark('ENTER') +"""Stream augmentation mark indicating that a selected element is being +entered.""" + +INSIDE = TransformMark('INSIDE') +"""Stream augmentation mark indicating that processing is currently inside a +selected element.""" + +OUTSIDE = TransformMark('OUTSIDE') +"""Stream augmentation mark indicating that a match occurred outside a selected +element.""" + +ATTR = TransformMark('ATTR') +"""Stream augmentation mark indicating a selected element attribute.""" + +EXIT = TransformMark('EXIT') +"""Stream augmentation mark indicating that a selected element is being +exited.""" + +BREAK = TransformMark('BREAK') +"""Stream augmentation mark indicating a break between two otherwise contiguous +blocks of marked events. + +This is used primarily by the cut() transform to provide later transforms with +an opportunity to operate on the cut buffer. +""" + + +class PushBackStream(object): + """Allows a single event to be pushed back onto the stream and re-consumed. + """ + def __init__(self, stream): + self.stream = iter(stream) + self.peek = None + + def push(self, event): + assert self.peek is None + self.peek = event + + def __iter__(self): + while True: + if self.peek is not None: + peek = self.peek + self.peek = None + yield peek + else: + try: + event = self.stream.next() + yield event + except StopIteration: + if self.peek is None: + raise + + +class Transformer(object): + """Stream filter that can apply a variety of different transformations to + a stream. + + This is achieved by selecting the events to be transformed using XPath, + then applying the transformations to the events matched by the path + expression. Each marked event is in the form (mark, (kind, data, pos)), + where mark can be any of `ENTER`, `INSIDE`, `EXIT`, `OUTSIDE`, or `None`. + + The first three marks match `START` and `END` events, and any events + contained `INSIDE` any selected XML/HTML element. A non-element match + outside a `START`/`END` container (e.g. ``text()``) will yield an `OUTSIDE` + mark. + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + + Transformations act on selected stream events matching an XPath expression. + Here's an example of removing some markup (the title, in this case) + selected by an expression: + + >>> print html | Transformer('head/title').remove() + <html><head/><body>Some <em>body</em> text.</body></html> + + Inserted content can be passed in the form of a string, or a markup event + stream, which includes streams generated programmatically via the + `builder` module: + + >>> from genshi.builder import tag + >>> print html | Transformer('body').prepend(tag.h1('Document Title')) + <html><head><title>Some Title</title></head><body><h1>Document + Title</h1>Some <em>body</em> text.</body></html> + + Each XPath expression determines the set of tags that will be acted upon by + subsequent transformations. In this example we select the ``<title>`` text, + copy it into a buffer, then select the ``<body>`` element and paste the + copied text into the body as ``<h1>`` enclosed text: + + >>> buffer = StreamBuffer() + >>> print html | Transformer('head/title/text()').copy(buffer) \\ + ... .end().select('body').prepend(tag.h1(buffer)) + <html><head><title>Some Title</title></head><body><h1>Some Title</h1>Some + <em>body</em> text.</body></html> + + Transformations can also be assigned and reused, although care must be + taken when using buffers, to ensure that buffers are cleared between + transforms: + + >>> emphasis = Transformer('body//em').attr('class', 'emphasis') + >>> print html | emphasis + <html><head><title>Some Title</title></head><body>Some <em + class="emphasis">body</em> text.</body></html> + """ + + __slots__ = ['transforms'] + + def __init__(self, path='.'): + """Construct a new transformation filter. + + :param path: an XPath expression (as string) or a `Path` instance + """ + self.transforms = [SelectTransformation(path)] + + def __call__(self, stream, keep_marks=False): + """Apply the transform filter to the marked stream. + + :param stream: the marked event stream to filter + :param keep_marks: Do not strip transformer selection marks from the + stream. Useful for testing. + :return: the transformed stream + :rtype: `Stream` + """ + transforms = self._mark(stream) + for link in self.transforms: + transforms = link(transforms) + if not keep_marks: + transforms = self._unmark(transforms) + return Stream(transforms, + serializer=getattr(stream, 'serializer', None)) + + def apply(self, function): + """Apply a transformation to the stream. + + Transformations can be chained, similar to stream filters. Any callable + accepting a marked stream can be used as a transform. + + As an example, here is a simple `TEXT` event upper-casing transform: + + >>> def upper(stream): + ... for mark, (kind, data, pos) in stream: + ... if mark and kind is TEXT: + ... yield mark, (kind, data.upper(), pos) + ... else: + ... yield mark, (kind, data, pos) + >>> short_stream = HTML('<body>Some <em>test</em> text</body>') + >>> print short_stream | Transformer('.//em/text()').apply(upper) + <body>Some <em>TEST</em> text</body> + """ + transformer = Transformer() + transformer.transforms = self.transforms[:] + if isinstance(function, Transformer): + transformer.transforms.extend(function.transforms) + else: + transformer.transforms.append(function) + return transformer + + #{ Selection operations + + def select(self, path): + """Mark events matching the given XPath expression, within the current + selection. + + >>> html = HTML('<body>Some <em>test</em> text</body>') + >>> print html | Transformer().select('.//em').trace() + (None, ('START', (QName(u'body'), Attrs()), (None, 1, 0))) + (None, ('TEXT', u'Some ', (None, 1, 6))) + ('ENTER', ('START', (QName(u'em'), Attrs()), (None, 1, 11))) + ('INSIDE', ('TEXT', u'test', (None, 1, 15))) + ('EXIT', ('END', QName(u'em'), (None, 1, 19))) + (None, ('TEXT', u' text', (None, 1, 24))) + (None, ('END', QName(u'body'), (None, 1, 29))) + <body>Some <em>test</em> text</body> + + :param path: an XPath expression (as string) or a `Path` instance + :return: the stream augmented by transformation marks + :rtype: `Transformer` + """ + return self.apply(SelectTransformation(path)) + + def invert(self): + """Invert selection so that marked events become unmarked, and vice + versa. + + Specificaly, all marks are converted to null marks, and all null marks + are converted to OUTSIDE marks. + + >>> html = HTML('<body>Some <em>test</em> text</body>') + >>> print html | Transformer('//em').invert().trace() + ('OUTSIDE', ('START', (QName(u'body'), Attrs()), (None, 1, 0))) + ('OUTSIDE', ('TEXT', u'Some ', (None, 1, 6))) + (None, ('START', (QName(u'em'), Attrs()), (None, 1, 11))) + (None, ('TEXT', u'test', (None, 1, 15))) + (None, ('END', QName(u'em'), (None, 1, 19))) + ('OUTSIDE', ('TEXT', u' text', (None, 1, 24))) + ('OUTSIDE', ('END', QName(u'body'), (None, 1, 29))) + <body>Some <em>test</em> text</body> + + :rtype: `Transformer` + """ + return self.apply(InvertTransformation()) + + def end(self): + """End current selection, allowing all events to be selected. + + Example: + + >>> html = HTML('<body>Some <em>test</em> text</body>') + >>> print html | Transformer('//em').end().trace() + ('OUTSIDE', ('START', (QName(u'body'), Attrs()), (None, 1, 0))) + ('OUTSIDE', ('TEXT', u'Some ', (None, 1, 6))) + ('OUTSIDE', ('START', (QName(u'em'), Attrs()), (None, 1, 11))) + ('OUTSIDE', ('TEXT', u'test', (None, 1, 15))) + ('OUTSIDE', ('END', QName(u'em'), (None, 1, 19))) + ('OUTSIDE', ('TEXT', u' text', (None, 1, 24))) + ('OUTSIDE', ('END', QName(u'body'), (None, 1, 29))) + <body>Some <em>test</em> text</body> + + :return: the stream augmented by transformation marks + :rtype: `Transformer` + """ + return self.apply(EndTransformation()) + + #{ Deletion operations + + def empty(self): + """Empty selected elements of all content. + + Example: + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('.//em').empty() + <html><head><title>Some Title</title></head><body>Some <em/> + text.</body></html> + + :rtype: `Transformer` + """ + return self.apply(EmptyTransformation()) + + def remove(self): + """Remove selection from the stream. + + Example: + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('.//em').remove() + <html><head><title>Some Title</title></head><body>Some + text.</body></html> + + :rtype: `Transformer` + """ + return self.apply(RemoveTransformation()) + + #{ Direct element operations + + def unwrap(self): + """Remove outermost enclosing elements from selection. + + Example: + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('.//em').unwrap() + <html><head><title>Some Title</title></head><body>Some body + text.</body></html> + + :rtype: `Transformer` + """ + return self.apply(UnwrapTransformation()) + + def wrap(self, element): + """Wrap selection in an element. + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('.//em').wrap('strong') + <html><head><title>Some Title</title></head><body>Some + <strong><em>body</em></strong> text.</body></html> + + :param element: either a tag name (as string) or an `Element` object + :rtype: `Transformer` + """ + return self.apply(WrapTransformation(element)) + + #{ Content insertion operations + + def replace(self, content): + """Replace selection with content. + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('.//title/text()').replace('New Title') + <html><head><title>New Title</title></head><body>Some <em>body</em> + text.</body></html> + + :param content: Either a callable, an iterable of events, or a string + to insert. + :rtype: `Transformer` + """ + return self.apply(ReplaceTransformation(content)) + + def before(self, content): + """Insert content before selection. + + In this example we insert the word 'emphasised' before the <em> opening + tag: + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('.//em').before('emphasised ') + <html><head><title>Some Title</title></head><body>Some emphasised + <em>body</em> text.</body></html> + + :param content: Either a callable, an iterable of events, or a string + to insert. + :rtype: `Transformer` + """ + return self.apply(BeforeTransformation(content)) + + def after(self, content): + """Insert content after selection. + + Here, we insert some text after the </em> closing tag: + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('.//em').after(' rock') + <html><head><title>Some Title</title></head><body>Some <em>body</em> + rock text.</body></html> + + :param content: Either a callable, an iterable of events, or a string + to insert. + :rtype: `Transformer` + """ + return self.apply(AfterTransformation(content)) + + def prepend(self, content): + """Insert content after the ENTER event of the selection. + + Inserting some new text at the start of the <body>: + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('.//body').prepend('Some new body text. ') + <html><head><title>Some Title</title></head><body>Some new body text. + Some <em>body</em> text.</body></html> + + :param content: Either a callable, an iterable of events, or a string + to insert. + :rtype: `Transformer` + """ + return self.apply(PrependTransformation(content)) + + def append(self, content): + """Insert content before the END event of the selection. + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('.//body').append(' Some new body text.') + <html><head><title>Some Title</title></head><body>Some <em>body</em> + text. Some new body text.</body></html> + + :param content: Either a callable, an iterable of events, or a string + to insert. + :rtype: `Transformer` + """ + return self.apply(AppendTransformation(content)) + + #{ Attribute manipulation + + def attr(self, name, value): + """Add, replace or delete an attribute on selected elements. + + If `value` evaulates to `None` the attribute will be deleted from the + element: + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em class="before">body</em> <em>text</em>.</body>' + ... '</html>') + >>> print html | Transformer('body/em').attr('class', None) + <html><head><title>Some Title</title></head><body>Some <em>body</em> + <em>text</em>.</body></html> + + Otherwise the attribute will be set to `value`: + + >>> print html | Transformer('body/em').attr('class', 'emphasis') + <html><head><title>Some Title</title></head><body>Some <em + class="emphasis">body</em> <em class="emphasis">text</em>.</body></html> + + If `value` is a callable it will be called with the attribute name and + the `START` event for the matching element. Its return value will then + be used to set the attribute: + + >>> def print_attr(name, event): + ... attrs = event[1][1] + ... print attrs + ... return attrs.get(name) + >>> print html | Transformer('body/em').attr('class', print_attr) + Attrs([(QName(u'class'), u'before')]) + Attrs() + <html><head><title>Some Title</title></head><body>Some <em + class="before">body</em> <em>text</em>.</body></html> + + :param name: the name of the attribute + :param value: the value that should be set for the attribute. + :rtype: `Transformer` + """ + return self.apply(AttrTransformation(name, value)) + + #{ Buffer operations + + def copy(self, buffer, accumulate=False): + """Copy selection into buffer. + + The buffer is replaced by each *contiguous* selection before being passed + to the next transformation. If accumulate=True, further selections will + be appended to the buffer rather than replacing it. + + >>> from genshi.builder import tag + >>> buffer = StreamBuffer() + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('head/title/text()').copy(buffer) \\ + ... .end().select('body').prepend(tag.h1(buffer)) + <html><head><title>Some Title</title></head><body><h1>Some + Title</h1>Some <em>body</em> text.</body></html> + + This example illustrates that only a single contiguous selection will + be buffered: + + >>> print html | Transformer('head/title/text()').copy(buffer) \\ + ... .end().select('body/em').copy(buffer).end().select('body') \\ + ... .prepend(tag.h1(buffer)) + <html><head><title>Some Title</title></head><body><h1>Some + Title</h1>Some <em>body</em> text.</body></html> + >>> print buffer + <em>body</em> + + Element attributes can also be copied for later use: + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body><em>Some</em> <em class="before">body</em>' + ... '<em>text</em>.</body></html>') + >>> buffer = StreamBuffer() + >>> def apply_attr(name, entry): + ... return list(buffer)[0][1][1].get('class') + >>> print html | Transformer('body/em[@class]/@class').copy(buffer) \\ + ... .end().buffer().select('body/em[not(@class)]') \\ + ... .attr('class', apply_attr) + <html><head><title>Some Title</title></head><body><em + class="before">Some</em> <em class="before">body</em><em + class="before">text</em>.</body></html> + + + :param buffer: the `StreamBuffer` in which the selection should be + stored + :rtype: `Transformer` + :note: Copy (and cut) copy each individual selected object into the + buffer before passing to the next transform. For example, the + XPath ``*|text()`` will select all elements and text, each + instance of which will be copied to the buffer individually + before passing to the next transform. This has implications for + how ``StreamBuffer`` objects can be used, so some + experimentation may be required. + + """ + return self.apply(CopyTransformation(buffer, accumulate)) + + def cut(self, buffer, accumulate=False): + """Copy selection into buffer and remove the selection from the stream. + + >>> from genshi.builder import tag + >>> buffer = StreamBuffer() + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('.//em/text()').cut(buffer) \\ + ... .end().select('.//em').after(tag.h1(buffer)) + <html><head><title>Some Title</title></head><body>Some + <em/><h1>body</h1> text.</body></html> + + Specifying accumulate=True, appends all selected intervals onto the + buffer. Combining this with the .buffer() operation allows us operate + on all copied events rather than per-segment. See the documentation on + buffer() for more information. + + :param buffer: the `StreamBuffer` in which the selection should be + stored + :rtype: `Transformer` + :note: this transformation will buffer the entire input stream + """ + return self.apply(CutTransformation(buffer, accumulate)) + + def buffer(self): + """Buffer the entire stream (can consume a considerable amount of + memory). + + Useful in conjunction with copy(accumulate=True) and + cut(accumulate=True) to ensure that all marked events in the entire + stream are copied to the buffer before further transformations are + applied. + + For example, to move all <note> elements inside a <notes> tag at the + top of the document: + + >>> doc = HTML('<doc><notes></notes><body>Some <note>one</note> ' + ... 'text <note>two</note>.</body></doc>') + >>> buffer = StreamBuffer() + >>> print doc | Transformer('body/note').cut(buffer, accumulate=True) \\ + ... .end().buffer().select('notes').prepend(buffer) + <doc><notes><note>one</note><note>two</note></notes><body>Some text + .</body></doc> + + """ + return self.apply(list) + + #{ Miscellaneous operations + + def filter(self, filter): + """Apply a normal stream filter to the selection. The filter is called + once for each contiguous block of marked events. + + >>> from genshi.filters.html import HTMLSanitizer + >>> html = HTML('<html><body>Some text<script>alert(document.cookie)' + ... '</script> and some more text</body></html>') + >>> print html | Transformer('body/*').filter(HTMLSanitizer()) + <html><body>Some text and some more text</body></html> + + :param filter: The stream filter to apply. + :rtype: `Transformer` + """ + return self.apply(FilterTransformation(filter)) + + def map(self, function, kind): + """Applies a function to the ``data`` element of events of ``kind`` in + the selection. + + >>> html = HTML('<html><head><title>Some Title</title></head>' + ... '<body>Some <em>body</em> text.</body></html>') + >>> print html | Transformer('head/title').map(unicode.upper, TEXT) + <html><head><title>SOME TITLE</title></head><body>Some <em>body</em> + text.</body></html> + + :param function: the function to apply + :param kind: the kind of event the function should be applied to + :rtype: `Transformer` + """ + return self.apply(MapTransformation(function, kind)) + + def substitute(self, pattern, replace, count=1): + """Replace text matching a regular expression. + + Refer to the documentation for ``re.sub()`` for details. + + >>> html = HTML('<html><body>Some text, some more text and ' + ... '<b>some bold text</b>\\n' + ... '<i>some italicised text</i></body></html>') + >>> print html | Transformer('body/b').substitute('(?i)some', 'SOME') + <html><body>Some text, some more text and <b>SOME bold text</b> + <i>some italicised text</i></body></html> + >>> tags = tag.html(tag.body('Some text, some more text and\\n', + ... Markup('<b>some bold text</b>'))) + >>> print tags.generate() | Transformer('body').substitute( + ... '(?i)some', 'SOME') + <html><body>SOME text, some more text and + <b>SOME bold text</b></body></html> + + :param pattern: A regular expression object or string. + :param replace: Replacement pattern. + :param count: Number of replacements to make in each text fragment. + :rtype: `Transformer` + """ + return self.apply(SubstituteTransformation(pattern, replace, count)) + + def rename(self, name): + """Rename matching elements. + + >>> html = HTML('<html><body>Some text, some more text and ' + ... '<b>some bold text</b></body></html>') + >>> print html | Transformer('body/b').rename('strong') + <html><body>Some text, some more text and <strong>some bold text</strong></body></html> + """ + return self.apply(RenameTransformation(name)) + + def trace(self, prefix='', fileobj=None): + """Print events as they pass through the transform. + + >>> html = HTML('<body>Some <em>test</em> text</body>') + >>> print html | Transformer('em').trace() + (None, ('START', (QName(u'body'), Attrs()), (None, 1, 0))) + (None, ('TEXT', u'Some ', (None, 1, 6))) + ('ENTER', ('START', (QName(u'em'), Attrs()), (None, 1, 11))) + ('INSIDE', ('TEXT', u'test', (None, 1, 15))) + ('EXIT', ('END', QName(u'em'), (None, 1, 19))) + (None, ('TEXT', u' text', (None, 1, 24))) + (None, ('END', QName(u'body'), (None, 1, 29))) + <body>Some <em>test</em> text</body> + + :param prefix: a string to prefix each event with in the output + :param fileobj: the writable file-like object to write to; defaults to + the standard output stream + :rtype: `Transformer` + """ + return self.apply(TraceTransformation(prefix, fileobj=fileobj)) + + # Internal methods + + def _mark(self, stream): + for event in stream: + yield OUTSIDE, event + + def _unmark(self, stream): + for mark, event in stream: + kind = event[0] + if not (kind is None or kind is ATTR or kind is BREAK): + yield event + + +class SelectTransformation(object): + """Select and mark events that match an XPath expression.""" + + def __init__(self, path): + """Create selection. + + :param path: an XPath expression (as string) or a `Path` object + """ + if not isinstance(path, Path): + path = Path(path) + self.path = path + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: the marked event stream to filter + """ + namespaces = {} + variables = {} + test = self.path.test() + stream = iter(stream) + next = stream.next + for mark, event in stream: + if mark is None: + yield mark, event + continue + result = test(event, namespaces, variables) + # XXX This is effectively genshi.core._ensure() for transform + # streams. + if result is True: + if event[0] is START: + yield ENTER, event + depth = 1 + while depth > 0: + mark, subevent = next() + if subevent[0] is START: + depth += 1 + elif subevent[0] is END: + depth -= 1 + if depth == 0: + yield EXIT, subevent + else: + yield INSIDE, subevent + test(subevent, namespaces, variables, updateonly=True) + else: + yield OUTSIDE, event + elif isinstance(result, Attrs): + # XXX Selected *attributes* are given a "kind" of None to + # indicate they are not really part of the stream. + yield ATTR, (ATTR, (QName(event[1][0] + '@*'), result), event[2]) + yield None, event + elif isinstance(result, tuple): + yield OUTSIDE, result + elif result: + # XXX Assume everything else is "text"? + yield None, (TEXT, unicode(result), (None, -1, -1)) + else: + yield None, event + + +class InvertTransformation(object): + """Invert selection so that marked events become unmarked, and vice versa. + + Specificaly, all input marks are converted to null marks, and all input + null marks are converted to OUTSIDE marks. + """ + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: the marked event stream to filter + """ + for mark, event in stream: + if mark: + yield None, event + else: + yield OUTSIDE, event + + +class EndTransformation(object): + """End the current selection.""" + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: the marked event stream to filter + """ + for mark, event in stream: + yield OUTSIDE, event + + +class EmptyTransformation(object): + """Empty selected elements of all content.""" + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: the marked event stream to filter + """ + for mark, event in stream: + yield mark, event + if mark is ENTER: + for mark, event in stream: + if mark is EXIT: + yield mark, event + break + + +class RemoveTransformation(object): + """Remove selection from the stream.""" + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: the marked event stream to filter + """ + for mark, event in stream: + if mark is None: + yield mark, event + + +class UnwrapTransformation(object): + """Remove outtermost enclosing elements from selection.""" + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: the marked event stream to filter + """ + for mark, event in stream: + if mark not in (ENTER, EXIT): + yield mark, event + + +class WrapTransformation(object): + """Wrap selection in an element.""" + + def __init__(self, element): + if isinstance(element, Element): + self.element = element + else: + self.element = Element(element) + + def __call__(self, stream): + for mark, event in stream: + if mark: + element = list(self.element.generate()) + for prefix in element[:-1]: + yield None, prefix + yield mark, event + start = mark + stopped = False + for mark, event in stream: + if start is ENTER and mark is EXIT: + yield mark, event + stopped = True + break + if not mark: + break + yield mark, event + else: + stopped = True + yield None, element[-1] + if not stopped: + yield mark, event + else: + yield mark, event + + +class TraceTransformation(object): + """Print events as they pass through the transform.""" + + def __init__(self, prefix='', fileobj=None): + """Trace constructor. + + :param prefix: text to prefix each traced line with. + :param fileobj: the writable file-like object to write to + """ + self.prefix = prefix + self.fileobj = fileobj or sys.stdout + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: the marked event stream to filter + """ + for event in stream: + print>>self.fileobj, self.prefix + str(event) + yield event + + +class FilterTransformation(object): + """Apply a normal stream filter to the selection. The filter is called once + for each selection.""" + + def __init__(self, filter): + """Create the transform. + + :param filter: The stream filter to apply. + """ + self.filter = filter + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: The marked event stream to filter + """ + def flush(queue): + if queue: + for event in self.filter(queue): + yield OUTSIDE, event + del queue[:] + + queue = [] + for mark, event in stream: + if mark is ENTER: + queue.append(event) + for mark, event in stream: + queue.append(event) + if mark is EXIT: + break + for queue_event in flush(queue): + yield queue_event + elif mark is OUTSIDE: + stopped = False + queue.append(event) + for mark, event in stream: + if mark is not OUTSIDE: + break + queue.append(event) + else: + stopped = True + for queue_event in flush(queue): + yield queue_event + if not stopped: + yield mark, event + else: + yield mark, event + for queue_event in flush(queue): + yield queue_event + + +class MapTransformation(object): + """Apply a function to the `data` element of events of ``kind`` in the + selection. + """ + + def __init__(self, function, kind): + """Create the transform. + + :param function: the function to apply; the function must take one + argument, the `data` element of each selected event + :param kind: the stream event ``kind`` to apply the `function` to + """ + self.function = function + self.kind = kind + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: The marked event stream to filter + """ + for mark, (kind, data, pos) in stream: + if mark and self.kind in (None, kind): + yield mark, (kind, self.function(data), pos) + else: + yield mark, (kind, data, pos) + + +class SubstituteTransformation(object): + """Replace text matching a regular expression. + + Refer to the documentation for ``re.sub()`` for details. + """ + def __init__(self, pattern, replace, count=0): + """Create the transform. + + :param pattern: A regular expression object, or string. + :param replace: Replacement pattern. + :param count: Number of replacements to make in each text fragment. + """ + if isinstance(pattern, basestring): + self.pattern = re.compile(pattern) + else: + self.pattern = pattern + self.count = count + self.replace = replace + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: The marked event stream to filter + """ + for mark, (kind, data, pos) in stream: + if mark is not None and kind is TEXT: + new_data = self.pattern.sub(self.replace, data, self.count) + if isinstance(data, Markup): + data = Markup(new_data) + else: + data = new_data + yield mark, (kind, data, pos) + + +class RenameTransformation(object): + """Rename matching elements.""" + def __init__(self, name): + """Create the transform. + + :param name: New element name. + """ + self.name = QName(name) + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: The marked event stream to filter + """ + for mark, (kind, data, pos) in stream: + if mark is ENTER: + data = self.name, data[1] + elif mark is EXIT: + data = self.name + yield mark, (kind, data, pos) + + +class InjectorTransformation(object): + """Abstract base class for transformations that inject content into a + stream. + + >>> class Top(InjectorTransformation): + ... def __call__(self, stream): + ... for event in self._inject(): + ... yield event + ... for event in stream: + ... yield event + >>> html = HTML('<body>Some <em>test</em> text</body>') + >>> print html | Transformer('.//em').apply(Top('Prefix ')) + Prefix <body>Some <em>test</em> text</body> + """ + def __init__(self, content): + """Create a new injector. + + :param content: An iterable of Genshi stream events, or a string to be + injected. + """ + self.content = content + + def _inject(self): + content = self.content + if hasattr(content, '__call__'): + content = content() + for event in _ensure(content): + yield None, event + + +class ReplaceTransformation(InjectorTransformation): + """Replace selection with content.""" + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: The marked event stream to filter + """ + stream = PushBackStream(stream) + for mark, event in stream: + if mark is not None: + start = mark + for subevent in self._inject(): + yield subevent + for mark, event in stream: + if start is ENTER: + if mark is EXIT: + break + elif mark != start: + stream.push((mark, event)) + break + else: + yield mark, event + + +class BeforeTransformation(InjectorTransformation): + """Insert content before selection.""" + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: The marked event stream to filter + """ + stream = PushBackStream(stream) + for mark, event in stream: + if mark is not None: + start = mark + for subevent in self._inject(): + yield subevent + yield mark, event + for mark, event in stream: + if mark != start and start is not ENTER: + stream.push((mark, event)) + break + yield mark, event + if start is ENTER and mark is EXIT: + break + else: + yield mark, event + + +class AfterTransformation(InjectorTransformation): + """Insert content after selection.""" + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: The marked event stream to filter + """ + stream = PushBackStream(stream) + for mark, event in stream: + yield mark, event + if mark: + start = mark + for mark, event in stream: + if start is not ENTER and mark != start: + stream.push((mark, event)) + break + yield mark, event + if start is ENTER and mark is EXIT: + break + for subevent in self._inject(): + yield subevent + + +class PrependTransformation(InjectorTransformation): + """Prepend content to the inside of selected elements.""" + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: The marked event stream to filter + """ + for mark, event in stream: + yield mark, event + if mark is ENTER: + for subevent in self._inject(): + yield subevent + + +class AppendTransformation(InjectorTransformation): + """Append content after the content of selected elements.""" + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: The marked event stream to filter + """ + for mark, event in stream: + yield mark, event + if mark is ENTER: + for mark, event in stream: + if mark is EXIT: + break + yield mark, event + for subevent in self._inject(): + yield subevent + yield mark, event + + +class AttrTransformation(object): + """Set an attribute on selected elements.""" + + def __init__(self, name, value): + """Construct transform. + + :param name: name of the attribute that should be set + :param value: the value to set + """ + self.name = name + self.value = value + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: The marked event stream to filter + """ + callable_value = hasattr(self.value, '__call__') + for mark, (kind, data, pos) in stream: + if mark is ENTER: + if callable_value: + value = self.value(self.name, (kind, data, pos)) + else: + value = self.value + if value is None: + attrs = data[1] - [QName(self.name)] + else: + attrs = data[1] | [(QName(self.name), value)] + data = (data[0], attrs) + yield mark, (kind, data, pos) + + + +class StreamBuffer(Stream): + """Stream event buffer used for cut and copy transformations.""" + + def __init__(self): + """Create the buffer.""" + Stream.__init__(self, []) + + def append(self, event): + """Add an event to the buffer. + + :param event: the markup event to add + """ + self.events.append(event) + + def reset(self): + """Empty the buffer of events.""" + del self.events[:] + + +class CopyTransformation(object): + """Copy selected events into a buffer for later insertion.""" + + def __init__(self, buffer, accumulate=False): + """Create the copy transformation. + + :param buffer: the `StreamBuffer` in which the selection should be + stored + """ + if not accumulate: + buffer.reset() + self.buffer = buffer + self.accumulate = accumulate + + def __call__(self, stream): + """Apply the transformation to the marked stream. + + :param stream: the marked event stream to filter + """ + stream = PushBackStream(stream) + + for mark, event in stream: + if mark: + if not self.accumulate: + self.buffer.reset() + events = [(mark, event)] + self.buffer.append(event) + start = mark + for mark, event in stream: + if start is not ENTER and mark != start: + stream.push((mark, event)) + break + events.append((mark, event)) + self.buffer.append(event) + if start is ENTER and mark is EXIT: + break + for i in events: + yield i + else: + yield mark, event + + +class CutTransformation(object): + """Cut selected events into a buffer for later insertion and remove the + selection. + """ + + def __init__(self, buffer, accumulate=False): + """Create the cut transformation. + + :param buffer: the `StreamBuffer` in which the selection should be + stored + """ + self.buffer = buffer + self.accumulate = accumulate + + + def __call__(self, stream): + """Apply the transform filter to the marked stream. + + :param stream: the marked event stream to filter + """ + attributes = [] + stream = PushBackStream(stream) + broken = False + if not self.accumulate: + self.buffer.reset() + for mark, event in stream: + if mark: + # Send a BREAK event if there was no other event sent between + if not self.accumulate: + if not broken and self.buffer: + yield BREAK, (BREAK, None, None) + self.buffer.reset() + self.buffer.append(event) + start = mark + if mark is ATTR: + attributes.extend([name for name, _ in event[1][1]]) + for mark, event in stream: + if start is mark is ATTR: + attributes.extend([name for name, _ in event[1][1]]) + # Handle non-element contiguous selection + if start is not ENTER and mark != start: + # Operating on the attributes of a START event + if start is ATTR: + kind, data, pos = event + assert kind is START + data = (data[0], data[1] - attributes) + attributes = None + stream.push((mark, (kind, data, pos))) + else: + stream.push((mark, event)) + break + self.buffer.append(event) + if start is ENTER and mark is EXIT: + break + broken = False + else: + broken = True + yield mark, event + if not broken and self.buffer: + yield BREAK, (BREAK, None, None)
--- a/genshi/input.py +++ b/genshi/input.py @@ -17,10 +17,6 @@ from itertools import chain from xml.parsers import expat -try: - frozenset -except NameError: - from sets import ImmutableSet as frozenset import HTMLParser as html import htmlentitydefs from StringIO import StringIO @@ -133,10 +129,6 @@ parser.UseForeignDTD() parser.ExternalEntityRefHandler = self._build_foreign - # Location reporting is only support in Python >= 2.4 - if not hasattr(parser, 'CurrentLineNumber'): - self._getpos = self._getpos_unknown - self.expat = parser self._queue = []
--- a/genshi/output.py +++ b/genshi/output.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -16,10 +16,6 @@ """ from itertools import chain -try: - frozenset -except NameError: - from sets import ImmutableSet as frozenset import re from genshi.core import escape, Attrs, Markup, Namespace, QName, StreamEventKind @@ -30,7 +26,7 @@ 'XHTMLSerializer', 'HTMLSerializer', 'TextSerializer'] __docformat__ = 'restructuredtext en' -def encode(iterator, method='xml', encoding='utf-8'): +def encode(iterator, method='xml', encoding='utf-8', out=None): """Encode serializer output into a string. :param iterator: the iterator returned from serializing a stream (basically @@ -39,16 +35,27 @@ representable in the specified encoding are treated :param encoding: how the output string should be encoded; if set to `None`, this method returns a `unicode` object - :return: a string or unicode object (depending on the `encoding` parameter) + :param out: a file-like object that the output should be written to + instead of being returned as one big string; note that if + this is a file or socket (or similar), the `encoding` must + not be `None` (that is, the output must be encoded) + :return: a `str` or `unicode` object (depending on the `encoding` + parameter), or `None` if the `out` parameter is provided + :since: version 0.4.1 + :note: Changed in 0.5: added the `out` parameter """ - output = u''.join(list(iterator)) if encoding is not None: errors = 'replace' if method != 'text' and not isinstance(method, TextSerializer): errors = 'xmlcharrefreplace' - return output.encode(encoding, errors) - return output + _encode = lambda string: string.encode(encoding, errors) + else: + _encode = lambda string: string + if out is None: + return _encode(u''.join(list(iterator))) + for chunk in iterator: + out.write(_encode(chunk)) def get_serializer(method='xml', **kwargs): """Return a serializer object for the given method. @@ -103,6 +110,25 @@ ) XHTML = XHTML_STRICT + XHTML11 = ( + 'html', '-//W3C//DTD XHTML 1.1//EN', + 'http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd' + ) + + SVG_FULL = ( + 'svg', '-//W3C//DTD SVG 1.1//EN', + 'http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd' + ) + SVG_BASIC = ( + 'svg', '-//W3C//DTD SVG Basic 1.1//EN', + 'http://www.w3.org/Graphics/SVG/1.1/DTD/svg11-basic.dtd' + ) + SVG_TINY = ( + 'svg', '-//W3C//DTD SVG Tiny 1.1//EN', + 'http://www.w3.org/Graphics/SVG/1.1/DTD/svg11-tiny.dtd' + ) + SVG = SVG_FULL + def get(cls, name): """Return the ``(name, pubid, sysid)`` tuple of the ``DOCTYPE`` declaration for the specified name. @@ -110,11 +136,15 @@ The following names are recognized in this version: * "html" or "html-strict" for the HTML 4.01 strict DTD * "html-transitional" for the HTML 4.01 transitional DTD - * "html-transitional" for the HTML 4.01 frameset DTD + * "html-frameset" for the HTML 4.01 frameset DTD * "html5" for the ``DOCTYPE`` proposed for HTML5 * "xhtml" or "xhtml-strict" for the XHTML 1.0 strict DTD * "xhtml-transitional" for the XHTML 1.0 transitional DTD * "xhtml-frameset" for the XHTML 1.0 frameset DTD + * "xhtml11" for the XHTML 1.1 DTD + * "svg" or "svg-full" for the SVG 1.1 DTD + * "svg-basic" for the SVG Basic 1.1 DTD + * "svg-tiny" for the SVG Tiny 1.1 DTD :param name: the name of the ``DOCTYPE`` :return: the ``(name, pubid, sysid)`` tuple for the requested @@ -129,6 +159,10 @@ 'xhtml': cls.XHTML, 'xhtml-strict': cls.XHTML_STRICT, 'xhtml-transitional': cls.XHTML_TRANSITIONAL, 'xhtml-frameset': cls.XHTML_FRAMESET, + 'xhtml11': cls.XHTML11, + 'svg': cls.SVG, 'svg-full': cls.SVG_FULL, + 'svg-basic': cls.SVG_BASIC, + 'svg-tiny': cls.SVG_TINY }.get(name.lower()) get = classmethod(get) @@ -156,21 +190,17 @@ stripped from the output :note: Changed in 0.4.2: The `doctype` parameter can now be a string. """ - self.preamble = [] - if doctype: - if isinstance(doctype, basestring): - doctype = DocType.get(doctype) - self.preamble.append((DOCTYPE, doctype, (None, -1, -1))) self.filters = [EmptyTagFilter()] if strip_whitespace: self.filters.append(WhitespaceFilter(self._PRESERVE_SPACE)) self.filters.append(NamespaceFlattener(prefixes=namespace_prefixes)) + if doctype: + self.filters.append(DocTypeInserter(doctype)) def __call__(self, stream): have_decl = have_doctype = False in_cdata = False - stream = chain(self.preamble, stream) for filter_ in self.filters: stream = filter_(stream) for kind, data, pos in stream: @@ -217,7 +247,7 @@ if sysid: buf.append(' "%s"') buf.append('>\n') - yield Markup(u''.join(buf), *filter(None, data)) + yield Markup(u''.join(buf)) % filter(None, data) have_doctype = True elif kind is START_CDATA: @@ -253,7 +283,7 @@ ]) def __init__(self, doctype=None, strip_whitespace=True, - namespace_prefixes=None): + namespace_prefixes=None, drop_xml_decl=True): super(XHTMLSerializer, self).__init__(doctype, False) self.filters = [EmptyTagFilter()] if strip_whitespace: @@ -261,14 +291,17 @@ namespace_prefixes = namespace_prefixes or {} namespace_prefixes['http://www.w3.org/1999/xhtml'] = '' self.filters.append(NamespaceFlattener(prefixes=namespace_prefixes)) + if doctype: + self.filters.append(DocTypeInserter(doctype)) + self.drop_xml_decl = drop_xml_decl def __call__(self, stream): boolean_attrs = self._BOOLEAN_ATTRS empty_elems = self._EMPTY_ELEMS - have_doctype = False + drop_xml_decl = self.drop_xml_decl + have_decl = have_doctype = False in_cdata = False - stream = chain(self.preamble, stream) for filter_ in self.filters: stream = filter_(stream) for kind, data, pos in stream: @@ -279,6 +312,10 @@ for attr, value in attrib: if attr in boolean_attrs: value = attr + elif attr == u'xml:lang' and u'lang' not in attrib: + buf += [' lang="', escape(value), '"'] + elif attr == u'xml:space': + continue buf += [' ', attr, '="', escape(value), '"'] if kind is EMPTY: if tag in empty_elems: @@ -311,9 +348,21 @@ if sysid: buf.append(' "%s"') buf.append('>\n') - yield Markup(u''.join(buf), *filter(None, data)) + yield Markup(u''.join(buf)) % filter(None, data) have_doctype = True + elif kind is XML_DECL and not have_decl and not drop_xml_decl: + version, encoding, standalone = data + buf = ['<?xml version="%s"' % version] + if encoding: + buf.append(' encoding="%s"' % encoding) + if standalone != -1: + standalone = standalone and 'yes' or 'no' + buf.append(' standalone="%s"' % standalone) + buf.append('?>\n') + yield Markup(u''.join(buf)) + have_decl = True + elif kind is START_CDATA: yield Markup('<![CDATA[') in_cdata = True @@ -354,7 +403,11 @@ if strip_whitespace: self.filters.append(WhitespaceFilter(self._PRESERVE_SPACE, self._NOESCAPE_ELEMS)) - self.filters.append(NamespaceStripper('http://www.w3.org/1999/xhtml')) + self.filters.append(NamespaceFlattener(prefixes={ + 'http://www.w3.org/1999/xhtml': '' + })) + if doctype: + self.filters.append(DocTypeInserter(doctype)) def __call__(self, stream): boolean_attrs = self._BOOLEAN_ATTRS @@ -363,7 +416,6 @@ have_doctype = False noescape = False - stream = chain(self.preamble, stream) for filter_ in self.filters: stream = filter_(stream) for kind, data, pos in stream: @@ -375,7 +427,10 @@ if attr in boolean_attrs: if value: buf += [' ', attr] - else: + elif ':' in attr: + if attr == 'xml:lang' and u'lang' not in attrib: + buf += [' lang="', escape(value), '"'] + elif attr != 'xmlns': buf += [' ', attr, '="', escape(value), '"'] buf.append('>') if kind is EMPTY: @@ -408,7 +463,7 @@ if sysid: buf.append(' "%s"') buf.append('>\n') - yield Markup(u''.join(buf), *filter(None, data)) + yield Markup(u''.join(buf)) % filter(None, data) have_doctype = True elif kind is PI: @@ -429,20 +484,34 @@ <Hello!> If text events contain literal markup (instances of the `Markup` class), - tags or entities are stripped from the output: + that markup is by default passed through unchanged: - >>> elem = tag.div(Markup('<a href="foo">Hello!</a><br/>')) - >>> print elem - <div><a href="foo">Hello!</a><br/></div> - >>> print ''.join(TextSerializer()(elem.generate())) - Hello! + >>> elem = tag.div(Markup('<a href="foo">Hello & Bye!</a><br/>')) + >>> print elem.generate().render(TextSerializer) + <a href="foo">Hello & Bye!</a><br/> + + You can use the ``strip_markup`` to change this behavior, so that tags and + entities are stripped from the output (or in the case of entities, + replaced with the equivalent character): + + >>> print elem.generate().render(TextSerializer, strip_markup=True) + Hello & Bye! """ + def __init__(self, strip_markup=False): + """Create the serializer. + + :param strip_markup: whether markup (tags and encoded characters) found + in the text should be removed + """ + self.strip_markup = strip_markup + def __call__(self, stream): + strip_markup = self.strip_markup for event in stream: if event[0] is TEXT: data = event[1] - if type(data) is Markup: + if strip_markup and type(data) is Markup: data = data.striptags().stripentities() yield unicode(data) @@ -586,60 +655,6 @@ yield kind, data, pos -class NamespaceStripper(object): - r"""Stream filter that removes all namespace information from a stream, and - optionally strips out all tags not in a given namespace. - - :param namespace: the URI of the namespace that should not be stripped. If - not set, only elements with no namespace are included in - the output. - - >>> from genshi.input import XML - >>> xml = XML('''<doc xmlns="NS1" xmlns:two="NS2"> - ... <two:item/> - ... </doc>''') - >>> for kind, data, pos in NamespaceStripper(Namespace('NS1'))(xml): - ... print kind, repr(data) - START (u'doc', Attrs()) - TEXT u'\n ' - TEXT u'\n' - END u'doc' - """ - - def __init__(self, namespace=None): - if namespace is not None: - self.namespace = Namespace(namespace) - else: - self.namespace = {} - - def __call__(self, stream): - namespace = self.namespace - - for kind, data, pos in stream: - - if kind is START or kind is EMPTY: - tag, attrs = data - if tag.namespace and tag not in namespace: - continue - - new_attrs = [] - for attr, value in attrs: - if not attr.namespace or attr in namespace: - new_attrs.append((attr, value)) - - data = tag.localname, Attrs(new_attrs) - - elif kind is END: - if data.namespace and data not in namespace: - continue - data = data.localname - - elif kind is START_NS or kind is END_NS: - continue - - yield kind, data, pos - - class WhitespaceFilter(object): """A filter that removes extraneous ignorable white space from the stream. @@ -714,3 +729,33 @@ if kind: yield kind, data, pos + + +class DocTypeInserter(object): + """A filter that inserts the DOCTYPE declaration in the correct location, + after the XML declaration. + """ + def __init__(self, doctype): + """Initialize the filter. + + :param doctype: DOCTYPE as a string or DocType object. + """ + if isinstance(doctype, basestring): + doctype = DocType.get(doctype) + self.doctype_event = (DOCTYPE, doctype, (None, -1, -1)) + + def __call__(self, stream): + doctype_inserted = False + for kind, data, pos in stream: + if not doctype_inserted: + doctype_inserted = True + if kind is XML_DECL: + yield (kind, data, pos) + yield self.doctype_event + continue + yield self.doctype_event + + yield (kind, data, pos) + + if not doctype_inserted: + yield self.doctype_event
--- a/genshi/path.py +++ b/genshi/path.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -15,27 +15,42 @@ >>> from genshi.input import XML >>> doc = XML('''<doc> -... <items count="2"> +... <items count="4"> ... <item status="new"> ... <summary>Foo</summary> ... </item> ... <item status="closed"> ... <summary>Bar</summary> ... </item> +... <item status="closed" resolution="invalid"> +... <summary>Baz</summary> +... </item> +... <item status="closed" resolution="fixed"> +... <summary>Waz</summary> +... </item> ... </items> ... </doc>''') ->>> print doc.select('items/item[@status="closed"]/summary/text()') -Bar +>>> print doc.select('items/item[@status="closed" and ' +... '(@resolution="invalid" or not(@resolution))]/summary/text()') +BarBaz Because the XPath engine operates on markup streams (as opposed to tree structures), it only implements a subset of the full XPath 1.0 language. """ +from collections import deque +try: + from functools import reduce +except ImportError: + pass # builtin in Python <= 2.5 from math import ceil, floor +import operator import re +from itertools import chain from genshi.core import Stream, Attrs, Namespace, QName -from genshi.core import START, END, TEXT, COMMENT, PI +from genshi.core import START, END, TEXT, START_NS, END_NS, COMMENT, PI, \ + START_CDATA, END_CDATA __all__ = ['Path', 'PathSyntaxError'] __docformat__ = 'restructuredtext en' @@ -65,14 +80,446 @@ SELF = Axis.SELF +class GenericStrategy(object): + + @classmethod + def supports(cls, path): + return True + + def __init__(self, path): + self.path = path + + def test(self, ignore_context): + p = self.path + if ignore_context: + if p[0][0] is ATTRIBUTE: + steps = [_DOTSLASHSLASH] + p + else: + steps = [(DESCENDANT_OR_SELF, p[0][1], p[0][2])] + p[1:] + elif p[0][0] is CHILD or p[0][0] is ATTRIBUTE \ + or p[0][0] is DESCENDANT: + steps = [_DOTSLASH] + p + else: + steps = p + + # for node it contains all positions of xpath expression + # where its child should start checking for matches + # with list of corresponding context counters + # there can be many of them, because position that is from + # descendant-like axis can be achieved from different nodes + # for example <a><a><b/></a></a> should match both //a//b[1] + # and //a//b[2] + # positions always form increasing sequence (invariant) + stack = [[(0, [[]])]] + + def _test(event, namespaces, variables, updateonly=False): + kind, data, pos = event[:3] + retval = None + + # Manage the stack that tells us "where we are" in the stream + if kind is END: + if stack: + stack.pop() + return None + if kind is START_NS or kind is END_NS \ + or kind is START_CDATA or kind is END_CDATA: + # should we make namespaces work? + return None + + pos_queue = deque([(pos, cou, []) for pos, cou in stack[-1]]) + next_pos = [] + + # length of real part of path - we omit attribute axis + real_len = len(steps) - ((steps[-1][0] == ATTRIBUTE) or 1 and 0) + last_checked = -1 + + # places where we have to check for match, are these + # provided by parent + while pos_queue: + x, pcou, mcou = pos_queue.popleft() + axis, nodetest, predicates = steps[x] + + # we need to push descendant-like positions from parent + # further + if (axis is DESCENDANT or axis is DESCENDANT_OR_SELF) and pcou: + if next_pos and next_pos[-1][0] == x: + next_pos[-1][1].extend(pcou) + else: + next_pos.append((x, pcou)) + + # nodetest first + if not nodetest(kind, data, pos, namespaces, variables): + continue + + # counters packs that were already bad + missed = set() + counters_len = len(pcou) + len(mcou) + + # number of counters - we have to create one + # for every context position based predicate + cnum = 0 + + # tells if we have match with position x + matched = True + + if predicates: + for predicate in predicates: + pretval = predicate(kind, data, pos, + namespaces, + variables) + if type(pretval) is float: # FIXME <- need to check + # this for other types that + # can be coerced to float + + # each counter pack needs to be checked + for i, cou in enumerate(chain(pcou, mcou)): + # it was bad before + if i in missed: + continue + + if len(cou) < cnum + 1: + cou.append(0) + cou[cnum] += 1 + + # it is bad now + if cou[cnum] != int(pretval): + missed.add(i) + + # none of counters pack was good + if len(missed) == counters_len: + pretval = False + cnum += 1 + + if not pretval: + matched = False + break + + if not matched: + continue + + # counter for next position with current node as context node + child_counter = [] + + if x + 1 == real_len: + # we reached end of expression, because x + 1 + # is equal to the length of expression + matched = True + axis, nodetest, predicates = steps[-1] + if axis is ATTRIBUTE: + matched = nodetest(kind, data, pos, namespaces, + variables) + if matched: + retval = matched + else: + next_axis = steps[x + 1][0] + + # if next axis allows matching self we have + # to add next position to our queue + if next_axis is DESCENDANT_OR_SELF or next_axis is SELF: + if not pos_queue or pos_queue[0][0] > x + 1: + pos_queue.appendleft((x + 1, [], [child_counter])) + else: + pos_queue[0][2].append(child_counter) + + # if axis is not self we have to add it to child's list + if next_axis is not SELF: + next_pos.append((x + 1, [child_counter])) + + if kind is START: + stack.append(next_pos) + + return retval + + return _test + + +class SimplePathStrategy(object): + """Strategy for path with only local names, attributes and text nodes.""" + + @classmethod + def supports(cls, path): + if path[0][0] is ATTRIBUTE: + return False + allowed_tests = (LocalNameTest, CommentNodeTest, TextNodeTest) + for _, nodetest, predicates in path: + if predicates: + return False + if not isinstance(nodetest, allowed_tests): + return False + return True + + def __init__(self, path): + # fragments is list of tuples (fragment, pi, attr, self_beginning) + # fragment is list of nodetests for fragment of path with only + # child:: axes between + # pi is KMP partial match table for this fragment + # attr is attribute nodetest if fragment ends with @ and None otherwise + # self_beginning is True if axis for first fragment element + # was self (first fragment) or descendant-or-self (farther fragment) + self.fragments = [] + + self_beginning = False + fragment = [] + + def nodes_equal(node1, node2): + """Tests if two node tests are equal""" + if node1.__class__ is not node2.__class__: + return False + if node1.__class__ == LocalNameTest: + return node1.name == node2.name + return True + + def calculate_pi(f): + """KMP prefix calculation for table""" + # the indexes in prefix table are shifted by one + # in comparision with common implementations + # pi[i] = NORMAL_PI[i + 1] + if len(f) == 0: + return [] + pi = [0] + s = 0 + for i in xrange(1, len(f)): + while s > 0 and not nodes_equal(f[s], f[i]): + s = pi[s-1] + if nodes_equal(f[s], f[i]): + s += 1 + pi.append(s) + return pi + + for axis in path: + if axis[0] is SELF: + if len(fragment) != 0: + # if element is not first in fragment it has to be + # the same as previous one + # for example child::a/self::b is always wrong + if axis[1] != fragment[-1][1]: + self.fragments = None + return + else: + self_beginning = True + fragment.append(axis[1]) + elif axis[0] is CHILD: + fragment.append(axis[1]) + elif axis[0] is ATTRIBUTE: + pi = calculate_pi(fragment) + self.fragments.append((fragment, pi, axis[1], self_beginning)) + # attribute has always to be at the end, so we can jump out + return + else: + pi = calculate_pi(fragment) + self.fragments.append((fragment, pi, None, self_beginning)) + fragment = [axis[1]] + if axis[0] is DESCENDANT: + self_beginning = False + else: # DESCENDANT_OR_SELF + self_beginning = True + pi = calculate_pi(fragment) + self.fragments.append((fragment, pi, None, self_beginning)) + + def test(self, ignore_context): + # stack of triples (fid, p, ic) + # fid is index of current fragment + # p is position in this fragment + # ic is if we ignore context in this fragment + stack = [] + stack_push = stack.append + stack_pop = stack.pop + frags = self.fragments + frags_len = len(frags) + + def _test(event, namespaces, variables, updateonly=False): + # expression found impossible during init + if frags is None: + return None + + kind, data, pos = event[:3] + + # skip events we don't care about + if kind is END: + if stack: + stack_pop() + return None + if kind is START_NS or kind is END_NS \ + or kind is START_CDATA or kind is END_CDATA: + return None + + if not stack: + # root node, nothing on stack, special case + fid = 0 + # skip empty fragments (there can be actually only one) + while not frags[fid][0]: + fid += 1 + p = 0 + # empty fragment means descendant node at beginning + ic = ignore_context or (fid > 0) + + # expression can match first node, if first axis is self::, + # descendant-or-self:: or if ignore_context is True and + # axis is not descendant:: + if not frags[fid][3] and (not ignore_context or fid > 0): + # axis is not self-beggining, we have to skip this node + stack_push((fid, p, ic)) + return None + else: + # take position of parent + fid, p, ic = stack[-1] + + if fid is not None and not ic: + # fragment not ignoring context - we can't jump back + frag, pi, attrib, _ = frags[fid] + frag_len = len(frag) + + if p == frag_len: + # that probably means empty first fragment + pass + elif frag[p](kind, data, pos, namespaces, variables): + # match, so we can go further + p += 1 + else: + # not matched, so there will be no match in subtree + fid, p = None, None + + if p == frag_len and fid + 1 != frags_len: + # we made it to end of fragment, we can go to following + fid += 1 + p = 0 + ic = True + + if fid is None: + # there was no match in fragment not ignoring context + if kind is START: + stack_push((fid, p, ic)) + return None + + if ic: + # we are in fragment ignoring context + while True: + frag, pi, attrib, _ = frags[fid] + frag_len = len(frag) + + # KMP new "character" + while p > 0 and (p >= frag_len or not \ + frag[p](kind, data, pos, namespaces, variables)): + p = pi[p-1] + if frag[p](kind, data, pos, namespaces, variables): + p += 1 + + if p == frag_len: + # end of fragment reached + if fid + 1 == frags_len: + # that was last fragment + break + else: + fid += 1 + p = 0 + ic = True + if not frags[fid][3]: + # next fragment not self-beginning + break + else: + break + + if kind is START: + # we have to put new position on stack, for children + + if not ic and fid + 1 == frags_len and p == frag_len: + # it is end of the only, not context ignoring fragment + # so there will be no matches in subtree + stack_push((None, None, ic)) + else: + stack_push((fid, p, ic)) + + # have we reached the end of the last fragment? + if fid + 1 == frags_len and p == frag_len: + if attrib: # attribute ended path, return value + return attrib(kind, data, pos, namespaces, variables) + return True + + return None + + return _test + + +class SingleStepStrategy(object): + + @classmethod + def supports(cls, path): + return len(path) == 1 + + def __init__(self, path): + self.path = path + + def test(self, ignore_context): + steps = self.path + if steps[0][0] is ATTRIBUTE: + steps = [_DOTSLASH] + steps + select_attr = steps[-1][0] is ATTRIBUTE and steps[-1][1] or None + + # for every position in expression stores counters' list + # it is used for position based predicates + counters = [] + depth = [0] + + def _test(event, namespaces, variables, updateonly=False): + kind, data, pos = event[:3] + + # Manage the stack that tells us "where we are" in the stream + if kind is END: + if not ignore_context: + depth[0] -= 1 + return None + elif kind is START_NS or kind is END_NS \ + or kind is START_CDATA or kind is END_CDATA: + # should we make namespaces work? + return None + + if not ignore_context: + outside = (steps[0][0] is SELF and depth[0] != 0) \ + or (steps[0][0] is CHILD and depth[0] != 1) \ + or (steps[0][0] is DESCENDANT and depth[0] < 1) + if kind is START: + depth[0] += 1 + if outside: + return None + + axis, nodetest, predicates = steps[0] + if not nodetest(kind, data, pos, namespaces, variables): + return None + + if predicates: + cnum = 0 + for predicate in predicates: + pretval = predicate(kind, data, pos, namespaces, variables) + if type(pretval) is float: # FIXME <- need to check this + # for other types that can be + # coerced to float + if len(counters) < cnum + 1: + counters.append(0) + counters[cnum] += 1 + if counters[cnum] != int(pretval): + pretval = False + cnum += 1 + if not pretval: + return None + + if select_attr: + return select_attr(kind, data, pos, namespaces, variables) + + return True + + return _test + + class Path(object): """Implements basic XPath support on streams. - Instances of this class represent a "compiled" XPath expression, and provide - methods for testing the path against a stream, as well as extracting a - substream matching that path. + Instances of this class represent a "compiled" XPath expression, and + provide methods for testing the path against a stream, as well as + extracting a substream matching that path. """ + STRATEGIES = (SingleStepStrategy, SimplePathStrategy, GenericStrategy) + def __init__(self, text, filename=None, lineno=-1): """Create the path object from a string. @@ -83,6 +530,14 @@ """ self.source = text self.paths = PathParser(text, filename, lineno).parse() + self.strategies = [] + for path in self.paths: + for strategy_class in self.STRATEGIES: + if strategy_class.supports(path): + self.strategies.append(strategy_class(path)) + break + else: + raise NotImplemented, "This path is not implemented" def __repr__(self): paths = [] @@ -120,26 +575,27 @@ if variables is None: variables = {} stream = iter(stream) - def _generate(): + def _generate(stream=stream, ns=namespaces, vs=variables): + next = stream.next test = self.test() for event in stream: - result = test(event, namespaces, variables) + result = test(event, ns, vs) if result is True: yield event if event[0] is START: depth = 1 while depth > 0: - subevent = stream.next() + subevent = next() if subevent[0] is START: depth += 1 elif subevent[0] is END: depth -= 1 yield subevent - test(subevent, namespaces, variables, - updateonly=True) + test(subevent, ns, vs, updateonly=True) elif result: yield result - return Stream(_generate()) + return Stream(_generate(), + serializer=getattr(stream, 'serializer', None)) def test(self, ignore_context=False): """Returns a function that can be used to track whether the path matches @@ -159,8 +615,9 @@ >>> from genshi.input import XML >>> xml = XML('<root><elem><child id="1"/></elem><child id="2"/></root>') >>> test = Path('child').test() + >>> namespaces, variables = {}, {} >>> for event in xml: - ... if test(event, {}, {}): + ... if test(event, namespaces, variables): ... print event[0], repr(event[1]) START (QName(u'child'), Attrs([(QName(u'id'), u'2')])) @@ -171,114 +628,18 @@ stream against the path :rtype: ``function`` """ - paths = [(p, len(p), [0], [], [0] * len(p)) for p in [ - (ignore_context and [_DOTSLASHSLASH] or []) + p for p in self.paths - ]] - - def _test(event, namespaces, variables, updateonly=False): - kind, data, pos = event[:3] - retval = None - for steps, size, cursors, cutoff, counter in paths: - # Manage the stack that tells us "where we are" in the stream - if kind is END: - if cursors: - cursors.pop() - continue - elif kind is START: - cursors.append(cursors and cursors[-1] or 0) - - if updateonly or retval or not cursors: - continue - cursor = cursors[-1] - depth = len(cursors) - - if cutoff and depth + int(kind is not START) > cutoff[0]: - continue - - ctxtnode = not ignore_context and kind is START \ - and depth == 2 - matched = None - while 1: - # Fetch the next location step - axis, nodetest, predicates = steps[cursor] - - # If this is the start event for the context node, and the - # axis of the location step doesn't include the current - # element, skip the test - if ctxtnode and (axis is CHILD or axis is DESCENDANT): - break - - # Is this the last step of the location path? - last_step = cursor + 1 == size - - # Perform the actual node test - matched = nodetest(kind, data, pos, namespaces, variables) - - # The node test matched - if matched: + tests = [s.test(ignore_context) for s in self.strategies] + if len(tests) == 1: + return tests[0] - # Check all the predicates for this step - if predicates: - for predicate in predicates: - pretval = predicate(kind, data, pos, namespaces, - variables) - if type(pretval) is float: - counter[cursor] += 1 - if counter[cursor] != int(pretval): - pretval = False - if not pretval: - matched = None - break - - # Both the node test and the predicates matched - if matched: - if last_step: - if not ctxtnode or kind is not START \ - or axis is ATTRIBUTE or axis is SELF: - retval = matched - elif not ctxtnode or axis is SELF \ - or axis is DESCENDANT_OR_SELF: - cursor += 1 - cursors[-1] = cursor - cutoff[:] = [] - - if kind is START: - if last_step and not (axis is DESCENDANT or - axis is DESCENDANT_OR_SELF): - cutoff[:] = [depth] - - elif steps[cursor][0] is ATTRIBUTE: - # If the axis of the next location step is the - # attribute axis, we need to move on to processing - # that step without waiting for the next markup - # event - continue - - # We're done with this step if it's the last step or the - # axis isn't "self" - if not matched or last_step or not ( - axis is SELF or axis is DESCENDANT_OR_SELF): - break - - if (retval or not matched) and kind is START and \ - not (axis is DESCENDANT or axis is DESCENDANT_OR_SELF): - # If this step is not a closure, it cannot be matched until - # the current element is closed... so we need to move the - # cursor back to the previous closure and retest that - # against the current element - backsteps = [(i, k, d, p) for i, (k, d, p) - in enumerate(steps[:cursor]) - if k is DESCENDANT or k is DESCENDANT_OR_SELF] - backsteps.reverse() - for cursor, axis, nodetest, predicates in backsteps: - if nodetest(kind, data, pos, namespaces, variables): - cutoff[:] = [] - break - cursors[-1] = cursor - + def _multi(event, namespaces, variables, updateonly=False): + retval = None + for test in tests: + val = test(event, namespaces, variables, updateonly=updateonly) + if retval is None: + retval = val return retval - - return _test + return _multi class PathSyntaxError(Exception): @@ -351,19 +712,28 @@ steps = [] while True: if self.cur_token.startswith('/'): - if self.cur_token == '//': + if not steps: + if self.cur_token == '//': + # hack to make //* match every node - also root + self.next_token() + axis, nodetest, predicates = self._location_step() + steps.append((DESCENDANT_OR_SELF, nodetest, + predicates)) + if self.at_end or not self.cur_token.startswith('/'): + break + continue + else: + raise PathSyntaxError('Absolute location paths not ' + 'supported', self.filename, + self.lineno) + elif self.cur_token == '//': steps.append((DESCENDANT_OR_SELF, NodeTest(), [])) - elif not steps: - raise PathSyntaxError('Absolute location paths not ' - 'supported', self.filename, - self.lineno) self.next_token() axis, nodetest, predicates = self._location_step() if not axis: axis = CHILD steps.append((axis, nodetest, predicates)) - if self.at_end or not self.cur_token.startswith('/'): break @@ -476,11 +846,24 @@ return expr def _relational_expr(self): - expr = self._primary_expr() + expr = self._sub_expr() while self.cur_token in ('>', '>=', '<', '>='): op = _operator_map[self.cur_token] self.next_token() - expr = op(expr, self._primary_expr()) + expr = op(expr, self._sub_expr()) + return expr + + def _sub_expr(self): + token = self.cur_token + if token != '(': + return self._primary_expr() + self.next_token() + expr = self._or_expr() + if self.cur_token != ')': + raise PathSyntaxError('Expected ")" to close sub-expression, ' + 'but found "%s"' % self.cur_token, + self.filename, self.lineno) + self.next_token() return expr def _primary_expr(self): @@ -490,7 +873,7 @@ return StringLiteral(token[1:-1]) elif token[0].isdigit() or token[0] == '.': self.next_token() - return NumberLiteral(float(token)) + return NumberLiteral(as_float(token)) elif token == '$': token = self.next_token() self.next_token() @@ -527,6 +910,35 @@ return cls(*args) +# Type coercion + +def as_scalar(value): + """Convert value to a scalar. If a single element Attrs() object is passed + the value of the single attribute will be returned.""" + if isinstance(value, Attrs): + assert len(value) == 1 + return value[0][1] + else: + return value + +def as_float(value): + # FIXME - if value is a bool it will be coerced to 0.0 and consequently + # compared as a float. This is probably not ideal. + return float(as_scalar(value)) + +def as_long(value): + return long(as_scalar(value)) + +def as_string(value): + value = as_scalar(value) + if value is False: + return u'' + return unicode(value) + +def as_bool(value): + return bool(as_scalar(value)) + + # Node tests class PrincipalTypeTest(object): @@ -572,7 +984,7 @@ def __call__(self, kind, data, pos, namespaces, variables): if kind is START: if self.principal_type is ATTRIBUTE and self.name in data[1]: - return data[1].get(self.name) + return Attrs([(self.name, data[1].get(self.name))]) else: return data[0].localname == self.name def __repr__(self): @@ -591,7 +1003,7 @@ qname = QName('%s}%s' % (namespaces.get(self.prefix), self.name)) if kind is START: if self.principal_type is ATTRIBUTE and qname in data[1]: - return data[1].get(qname) + return Attrs([(self.name, data[1].get(self.name))]) else: return data[0] == qname def __repr__(self): @@ -650,11 +1062,12 @@ value. """ __slots__ = ['expr'] + _return_type = bool def __init__(self, expr): self.expr = expr def __call__(self, kind, data, pos, namespaces, variables): val = self.expr(kind, data, pos, namespaces, variables) - return bool(val) + return as_bool(val) def __repr__(self): return 'boolean(%r)' % self.expr @@ -667,7 +1080,7 @@ self.number = number def __call__(self, kind, data, pos, namespaces, variables): number = self.number(kind, data, pos, namespaces, variables) - return ceil(float(number)) + return ceil(as_float(number)) def __repr__(self): return 'ceiling(%r)' % self.number @@ -682,7 +1095,7 @@ strings = [] for item in [expr(kind, data, pos, namespaces, variables) for expr in self.exprs]: - strings.append(item) + strings.append(as_string(item)) return u''.join(strings) def __repr__(self): return 'concat(%s)' % ', '.join([repr(expr) for expr in self.exprs]) @@ -698,7 +1111,28 @@ def __call__(self, kind, data, pos, namespaces, variables): string1 = self.string1(kind, data, pos, namespaces, variables) string2 = self.string2(kind, data, pos, namespaces, variables) - return string2 in string1 + return as_string(string2) in as_string(string1) + def __repr__(self): + return 'contains(%r, %r)' % (self.string1, self.string2) + +class MatchesFunction(Function): + """The `matches` function, which returns whether a string matches a regular + expression. + """ + __slots__ = ['string1', 'string2'] + flag_mapping = {'s': re.S, 'm': re.M, 'i': re.I, 'x': re.X} + + def __init__(self, string1, string2, flags=''): + self.string1 = string1 + self.string2 = string2 + self.flags = self._map_flags(flags) + def __call__(self, kind, data, pos, namespaces, variables): + string1 = as_string(self.string1(kind, data, pos, namespaces, variables)) + string2 = as_string(self.string2(kind, data, pos, namespaces, variables)) + return re.search(string2, string1, self.flags) + def _map_flags(self, flags): + return reduce(operator.or_, + [self.flag_map[flag] for flag in flags], re.U) def __repr__(self): return 'contains(%r, %r)' % (self.string1, self.string2) @@ -719,7 +1153,7 @@ self.number = number def __call__(self, kind, data, pos, namespaces, variables): number = self.number(kind, data, pos, namespaces, variables) - return floor(float(number)) + return floor(as_float(number)) def __repr__(self): return 'floor(%r)' % self.number @@ -764,7 +1198,7 @@ def __init__(self, expr): self.expr = expr def __call__(self, kind, data, pos, namespaces, variables): - return not self.expr(kind, data, pos, namespaces, variables) + return not as_bool(self.expr(kind, data, pos, namespaces, variables)) def __repr__(self): return 'not(%s)' % self.expr @@ -779,7 +1213,7 @@ self.expr = expr def __call__(self, kind, data, pos, namespaces, variables): string = self.expr(kind, data, pos, namespaces, variables) - return self._normalize(' ', string.strip()) + return self._normalize(' ', as_string(string).strip()) def __repr__(self): return 'normalize-space(%s)' % repr(self.expr) @@ -790,7 +1224,7 @@ self.expr = expr def __call__(self, kind, data, pos, namespaces, variables): val = self.expr(kind, data, pos, namespaces, variables) - return float(val) + return as_float(val) def __repr__(self): return 'number(%r)' % self.expr @@ -803,7 +1237,7 @@ self.number = number def __call__(self, kind, data, pos, namespaces, variables): number = self.number(kind, data, pos, namespaces, variables) - return round(float(number)) + return round(as_float(number)) def __repr__(self): return 'round(%r)' % self.number @@ -818,7 +1252,7 @@ def __call__(self, kind, data, pos, namespaces, variables): string1 = self.string1(kind, data, pos, namespaces, variables) string2 = self.string2(kind, data, pos, namespaces, variables) - return string1.startswith(string2) + return as_string(string1).startswith(as_string(string2)) def __repr__(self): return 'starts-with(%r, %r)' % (self.string1, self.string2) @@ -831,7 +1265,7 @@ self.expr = expr def __call__(self, kind, data, pos, namespaces, variables): string = self.expr(kind, data, pos, namespaces, variables) - return len(string) + return len(as_string(string)) def __repr__(self): return 'string-length(%r)' % self.expr @@ -850,7 +1284,7 @@ length = 0 if self.length is not None: length = self.length(kind, data, pos, namespaces, variables) - return string[int(start):len(string) - int(length)] + return string[as_long(start):len(as_string(string)) - as_long(length)] def __repr__(self): if self.length is not None: return 'substring(%r, %r, %r)' % (self.string, self.start, @@ -867,8 +1301,8 @@ self.string1 = string1 self.string2 = string2 def __call__(self, kind, data, pos, namespaces, variables): - string1 = self.string1(kind, data, pos, namespaces, variables) - string2 = self.string2(kind, data, pos, namespaces, variables) + string1 = as_string(self.string1(kind, data, pos, namespaces, variables)) + string2 = as_string(self.string2(kind, data, pos, namespaces, variables)) index = string1.find(string2) if index >= 0: return string1[index + len(string2):] @@ -885,8 +1319,8 @@ self.string1 = string1 self.string2 = string2 def __call__(self, kind, data, pos, namespaces, variables): - string1 = self.string1(kind, data, pos, namespaces, variables) - string2 = self.string2(kind, data, pos, namespaces, variables) + string1 = as_string(self.string1(kind, data, pos, namespaces, variables)) + string2 = as_string(self.string2(kind, data, pos, namespaces, variables)) index = string1.find(string2) if index >= 0: return string1[:index] @@ -904,9 +1338,9 @@ self.fromchars = fromchars self.tochars = tochars def __call__(self, kind, data, pos, namespaces, variables): - string = self.string(kind, data, pos, namespaces, variables) - fromchars = self.fromchars(kind, data, pos, namespaces, variables) - tochars = self.tochars(kind, data, pos, namespaces, variables) + string = as_string(self.string(kind, data, pos, namespaces, variables)) + fromchars = as_string(self.fromchars(kind, data, pos, namespaces, variables)) + tochars = as_string(self.tochars(kind, data, pos, namespaces, variables)) table = dict(zip([ord(c) for c in fromchars], [ord(c) for c in tochars])) return string.translate(table) @@ -924,17 +1358,16 @@ _function_map = {'boolean': BooleanFunction, 'ceiling': CeilingFunction, 'concat': ConcatFunction, 'contains': ContainsFunction, - 'false': FalseFunction, 'floor': FloorFunction, - 'local-name': LocalNameFunction, 'name': NameFunction, - 'namespace-uri': NamespaceUriFunction, + 'matches': MatchesFunction, 'false': FalseFunction, 'floor': + FloorFunction, 'local-name': LocalNameFunction, 'name': + NameFunction, 'namespace-uri': NamespaceUriFunction, 'normalize-space': NormalizeSpaceFunction, 'not': NotFunction, 'number': NumberFunction, 'round': RoundFunction, - 'starts-with': StartsWithFunction, - 'string-length': StringLengthFunction, - 'substring': SubstringFunction, - 'substring-after': SubstringAfterFunction, - 'substring-before': SubstringBeforeFunction, - 'translate': TranslateFunction, 'true': TrueFunction} + 'starts-with': StartsWithFunction, 'string-length': + StringLengthFunction, 'substring': SubstringFunction, + 'substring-after': SubstringAfterFunction, 'substring-before': + SubstringBeforeFunction, 'translate': TranslateFunction, + 'true': TrueFunction} # Literals & Variables @@ -980,11 +1413,11 @@ self.lval = lval self.rval = rval def __call__(self, kind, data, pos, namespaces, variables): - lval = self.lval(kind, data, pos, namespaces, variables) + lval = as_bool(self.lval(kind, data, pos, namespaces, variables)) if not lval: return False rval = self.rval(kind, data, pos, namespaces, variables) - return bool(rval) + return as_bool(rval) def __repr__(self): return '%s and %s' % (self.lval, self.rval) @@ -995,8 +1428,8 @@ self.lval = lval self.rval = rval def __call__(self, kind, data, pos, namespaces, variables): - lval = self.lval(kind, data, pos, namespaces, variables) - rval = self.rval(kind, data, pos, namespaces, variables) + lval = as_scalar(self.lval(kind, data, pos, namespaces, variables)) + rval = as_scalar(self.rval(kind, data, pos, namespaces, variables)) return lval == rval def __repr__(self): return '%s=%s' % (self.lval, self.rval) @@ -1008,8 +1441,8 @@ self.lval = lval self.rval = rval def __call__(self, kind, data, pos, namespaces, variables): - lval = self.lval(kind, data, pos, namespaces, variables) - rval = self.rval(kind, data, pos, namespaces, variables) + lval = as_scalar(self.lval(kind, data, pos, namespaces, variables)) + rval = as_scalar(self.rval(kind, data, pos, namespaces, variables)) return lval != rval def __repr__(self): return '%s!=%s' % (self.lval, self.rval) @@ -1021,11 +1454,11 @@ self.lval = lval self.rval = rval def __call__(self, kind, data, pos, namespaces, variables): - lval = self.lval(kind, data, pos, namespaces, variables) + lval = as_bool(self.lval(kind, data, pos, namespaces, variables)) if lval: return True rval = self.rval(kind, data, pos, namespaces, variables) - return bool(rval) + return as_bool(rval) def __repr__(self): return '%s or %s' % (self.lval, self.rval) @@ -1038,7 +1471,7 @@ def __call__(self, kind, data, pos, namespaces, variables): lval = self.lval(kind, data, pos, namespaces, variables) rval = self.rval(kind, data, pos, namespaces, variables) - return float(lval) > float(rval) + return as_float(lval) > as_float(rval) def __repr__(self): return '%s>%s' % (self.lval, self.rval) @@ -1051,7 +1484,7 @@ def __call__(self, kind, data, pos, namespaces, variables): lval = self.lval(kind, data, pos, namespaces, variables) rval = self.rval(kind, data, pos, namespaces, variables) - return float(lval) >= float(rval) + return as_float(lval) >= as_float(rval) def __repr__(self): return '%s>=%s' % (self.lval, self.rval) @@ -1064,7 +1497,7 @@ def __call__(self, kind, data, pos, namespaces, variables): lval = self.lval(kind, data, pos, namespaces, variables) rval = self.rval(kind, data, pos, namespaces, variables) - return float(lval) < float(rval) + return as_float(lval) < as_float(rval) def __repr__(self): return '%s<%s' % (self.lval, self.rval) @@ -1077,7 +1510,7 @@ def __call__(self, kind, data, pos, namespaces, variables): lval = self.lval(kind, data, pos, namespaces, variables) rval = self.rval(kind, data, pos, namespaces, variables) - return float(lval) <= float(rval) + return as_float(lval) <= as_float(rval) def __repr__(self): return '%s<=%s' % (self.lval, self.rval) @@ -1087,3 +1520,4 @@ _DOTSLASHSLASH = (DESCENDANT_OR_SELF, PrincipalTypeTest(None), ()) +_DOTSLASH = (SELF, PrincipalTypeTest(None), ())
--- a/genshi/template/__init__.py +++ b/genshi/template/__init__.py @@ -18,6 +18,6 @@ BadDirectiveError from genshi.template.loader import TemplateLoader, TemplateNotFound from genshi.template.markup import MarkupTemplate -from genshi.template.text import TextTemplate +from genshi.template.text import TextTemplate, OldTextTemplate, NewTextTemplate __docformat__ = 'restructuredtext en'
new file mode 100644 --- /dev/null +++ b/genshi/template/_ast24.py @@ -0,0 +1,446 @@ +# Generated automatically, please do not edit +# Generator can be found in Genshi SVN, scripts/ast-generator.py + +__version__ = 43614 + +class AST(object): + _fields = None + __doc__ = None + +class operator(AST): + _fields = None + __doc__ = None + _attributes = [] +class Add(operator): + _fields = None + __doc__ = None + +class boolop(AST): + _fields = None + __doc__ = None + _attributes = [] +class And(boolop): + _fields = None + __doc__ = None + +class stmt(AST): + _fields = None + __doc__ = None + _attributes = ['lineno', 'col_offset'] +class Assert(stmt): + _fields = ('test', 'msg') + __doc__ = None + +class Assign(stmt): + _fields = ('targets', 'value') + __doc__ = None + +class expr(AST): + _fields = None + __doc__ = None + _attributes = ['lineno', 'col_offset'] +class Attribute(expr): + _fields = ('value', 'attr', 'ctx') + __doc__ = None + +class AugAssign(stmt): + _fields = ('target', 'op', 'value') + __doc__ = None + +class expr_context(AST): + _fields = None + __doc__ = None + _attributes = [] +class AugLoad(expr_context): + _fields = None + __doc__ = None + +class AugStore(expr_context): + _fields = None + __doc__ = None + +class BinOp(expr): + _fields = ('left', 'op', 'right') + __doc__ = None + +class BitAnd(operator): + _fields = None + __doc__ = None + +class BitOr(operator): + _fields = None + __doc__ = None + +class BitXor(operator): + _fields = None + __doc__ = None + +class BoolOp(expr): + _fields = ('op', 'values') + __doc__ = None + +class Break(stmt): + _fields = None + __doc__ = None + +class Call(expr): + _fields = ('func', 'args', 'keywords', 'starargs', 'kwargs') + __doc__ = None + +class ClassDef(stmt): + _fields = ('name', 'bases', 'body') + __doc__ = None + +class Compare(expr): + _fields = ('left', 'ops', 'comparators') + __doc__ = None + +class Continue(stmt): + _fields = None + __doc__ = None + +class Del(expr_context): + _fields = None + __doc__ = None + +class Delete(stmt): + _fields = ('targets',) + __doc__ = None + +class Dict(expr): + _fields = ('keys', 'values') + __doc__ = None + +class Div(operator): + _fields = None + __doc__ = None + +class slice(AST): + _fields = None + __doc__ = None + _attributes = [] +class Ellipsis(slice): + _fields = None + __doc__ = None + +class cmpop(AST): + _fields = None + __doc__ = None + _attributes = [] +class Eq(cmpop): + _fields = None + __doc__ = None + +class Exec(stmt): + _fields = ('body', 'globals', 'locals') + __doc__ = None + +class Expr(stmt): + _fields = ('value',) + __doc__ = None + +class mod(AST): + _fields = None + __doc__ = None + _attributes = [] +class Expression(mod): + _fields = ('body',) + __doc__ = None + +class ExtSlice(slice): + _fields = ('dims',) + __doc__ = None + +class FloorDiv(operator): + _fields = None + __doc__ = None + +class For(stmt): + _fields = ('target', 'iter', 'body', 'orelse') + __doc__ = None + +class FunctionDef(stmt): + _fields = ('name', 'args', 'body', 'decorators') + __doc__ = None + +class GeneratorExp(expr): + _fields = ('elt', 'generators') + __doc__ = None + +class Global(stmt): + _fields = ('names',) + __doc__ = None + +class Gt(cmpop): + _fields = None + __doc__ = None + +class GtE(cmpop): + _fields = None + __doc__ = None + +class If(stmt): + _fields = ('test', 'body', 'orelse') + __doc__ = None + +class IfExp(expr): + _fields = ('test', 'body', 'orelse') + __doc__ = None + +class Import(stmt): + _fields = ('names',) + __doc__ = None + +class ImportFrom(stmt): + _fields = ('module', 'names', 'level') + __doc__ = None + +class In(cmpop): + _fields = None + __doc__ = None + +class Index(slice): + _fields = ('value',) + __doc__ = None + +class Interactive(mod): + _fields = ('body',) + __doc__ = None + +class unaryop(AST): + _fields = None + __doc__ = None + _attributes = [] +class Invert(unaryop): + _fields = None + __doc__ = None + +class Is(cmpop): + _fields = None + __doc__ = None + +class IsNot(cmpop): + _fields = None + __doc__ = None + +class LShift(operator): + _fields = None + __doc__ = None + +class Lambda(expr): + _fields = ('args', 'body') + __doc__ = None + +class List(expr): + _fields = ('elts', 'ctx') + __doc__ = None + +class ListComp(expr): + _fields = ('elt', 'generators') + __doc__ = None + +class Load(expr_context): + _fields = None + __doc__ = None + +class Lt(cmpop): + _fields = None + __doc__ = None + +class LtE(cmpop): + _fields = None + __doc__ = None + +class Mod(operator): + _fields = None + __doc__ = None + +class Module(mod): + _fields = ('body',) + __doc__ = None + +class Mult(operator): + _fields = None + __doc__ = None + +class Name(expr): + _fields = ('id', 'ctx') + __doc__ = None + +class Not(unaryop): + _fields = None + __doc__ = None + +class NotEq(cmpop): + _fields = None + __doc__ = None + +class NotIn(cmpop): + _fields = None + __doc__ = None + +class Num(expr): + _fields = ('n',) + __doc__ = None + +class Or(boolop): + _fields = None + __doc__ = None + +class Param(expr_context): + _fields = None + __doc__ = None + +class Pass(stmt): + _fields = None + __doc__ = None + +class Pow(operator): + _fields = None + __doc__ = None + +class Print(stmt): + _fields = ('dest', 'values', 'nl') + __doc__ = None + +class RShift(operator): + _fields = None + __doc__ = None + +class Raise(stmt): + _fields = ('type', 'inst', 'tback') + __doc__ = None + +class Repr(expr): + _fields = ('value',) + __doc__ = None + +class Return(stmt): + _fields = ('value',) + __doc__ = None + +class Slice(slice): + _fields = ('lower', 'upper', 'step') + __doc__ = None + +class Store(expr_context): + _fields = None + __doc__ = None + +class Str(expr): + _fields = ('s',) + __doc__ = None + +class Sub(operator): + _fields = None + __doc__ = None + +class Subscript(expr): + _fields = ('value', 'slice', 'ctx') + __doc__ = None + +class Suite(mod): + _fields = ('body',) + __doc__ = None + +class TryExcept(stmt): + _fields = ('body', 'handlers', 'orelse') + __doc__ = None + +class TryFinally(stmt): + _fields = ('body', 'finalbody') + __doc__ = None + +class Tuple(expr): + _fields = ('elts', 'ctx') + __doc__ = None + +class UAdd(unaryop): + _fields = None + __doc__ = None + +class USub(unaryop): + _fields = None + __doc__ = None + +class UnaryOp(expr): + _fields = ('op', 'operand') + __doc__ = None + +class While(stmt): + _fields = ('test', 'body', 'orelse') + __doc__ = None + +class With(stmt): + _fields = ('context_expr', 'optional_vars', 'body') + __doc__ = None + +class Yield(expr): + _fields = ('value',) + __doc__ = None + +class alias(AST): + _fields = ('name', 'asname') + __doc__ = None + +class arguments(AST): + _fields = ('args', 'vararg', 'kwarg', 'defaults') + __doc__ = None + +class boolop(AST): + _fields = None + __doc__ = None + _attributes = [] + +class cmpop(AST): + _fields = None + __doc__ = None + _attributes = [] + +class comprehension(AST): + _fields = ('target', 'iter', 'ifs') + __doc__ = None + +class excepthandler(AST): + _fields = ('type', 'name', 'body', 'lineno', 'col_offset') + __doc__ = None + +class expr(AST): + _fields = None + __doc__ = None + _attributes = ['lineno', 'col_offset'] + +class expr_context(AST): + _fields = None + __doc__ = None + _attributes = [] + +class keyword(AST): + _fields = ('arg', 'value') + __doc__ = None + +class mod(AST): + _fields = None + __doc__ = None + _attributes = [] + +class operator(AST): + _fields = None + __doc__ = None + _attributes = [] + +class slice(AST): + _fields = None + __doc__ = None + _attributes = [] + +class stmt(AST): + _fields = None + __doc__ = None + _attributes = ['lineno', 'col_offset'] + +class unaryop(AST): + _fields = None + __doc__ = None + _attributes = [] +
new file mode 100644 --- /dev/null +++ b/genshi/template/ast24.py @@ -0,0 +1,506 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2008 Edgewall Software +# All rights reserved. +# +# This software is licensed as described in the file COPYING, which +# you should have received as part of this distribution. The terms +# are also available at http://genshi.edgewall.org/wiki/License. +# +# This software consists of voluntary contributions made by many +# individuals. For the exact contribution history, see the revision +# history and logs, available at http://genshi.edgewall.org/log/. + +"""Emulation of the proper abstract syntax tree API for Python 2.4.""" + +import compiler +import compiler.ast + +from genshi.template import _ast24 as _ast + +__all__ = ['_ast', 'parse'] +__docformat__ = 'restructuredtext en' + + +def _new(cls, *args, **kwargs): + ret = cls() + if ret._fields: + for attr, value in zip(ret._fields, args): + if attr in kwargs: + raise ValueError, 'Field set both in args and kwargs' + setattr(ret, attr, value) + for attr in kwargs: + if (getattr(ret, '_fields', None) and attr in ret._fields) \ + or (getattr(ret, '_attributes', None) and + attr in ret._attributes): + setattr(ret, attr, kwargs[attr]) + return ret + + +class ASTUpgrader(object): + """Transformer changing structure of Python 2.4 ASTs to + Python 2.5 ones. + + Transforms ``compiler.ast`` Abstract Syntax Tree to builtin ``_ast``. + It can use fake`` _ast`` classes and this way allow ``_ast`` emulation + in Python 2.4. + """ + + def __init__(self): + self.out_flags = None + self.lines = [-1] + + def _new(self, *args, **kwargs): + return _new(lineno = self.lines[-1], *args, **kwargs) + + def visit(self, node): + if node is None: + return None + if type(node) is tuple: + return tuple([self.visit(n) for n in node]) + lno = getattr(node, 'lineno', None) + if lno is not None: + self.lines.append(lno) + visitor = getattr(self, 'visit_%s' % node.__class__.__name__, None) + if visitor is None: + raise Exception('Unhandled node type %r' % type(node)) + + retval = visitor(node) + if lno is not None: + self.lines.pop() + return retval + + def visit_Module(self, node): + body = self.visit(node.node) + if node.doc: + body = [self._new(_ast.Expr, self._new(_ast.Str, node.doc))] + body + return self._new(_ast.Module, body) + + def visit_Expression(self, node): + return self._new(_ast.Expression, self.visit(node.node)) + + def _extract_args(self, node): + tab = node.argnames[:] + if node.flags & compiler.ast.CO_VARKEYWORDS: + kwarg = tab[-1] + tab = tab[:-1] + else: + kwarg = None + + if node.flags & compiler.ast.CO_VARARGS: + vararg = tab[-1] + tab = tab[:-1] + else: + vararg = None + + def _tup(t): + if isinstance(t, str): + return self._new(_ast.Name, t, _ast.Store()) + elif isinstance(t, tuple): + elts = [_tup(x) for x in t] + return self._new(_ast.Tuple, elts, _ast.Store()) + else: + raise NotImplemented + + args = [] + for arg in tab: + if isinstance(arg, str): + args.append(self._new(_ast.Name, arg, _ast.Param())) + elif isinstance(arg, tuple): + args.append(_tup(arg)) + else: + assert False, node.__class__ + + defaults = map(self.visit, node.defaults) + return self._new(_ast.arguments, args, vararg, kwarg, defaults) + + + def visit_Function(self, node): + if getattr(node, 'decorators', ()): + decorators = [self.visit(d) for d in node.decorators.nodes] + else: + decorators = [] + + args = self._extract_args(node) + body = self.visit(node.code) + if node.doc: + body = [self._new(_ast.Expr, self._new(_ast.Str, node.doc))] + body + return self._new(_ast.FunctionDef, node.name, args, body, decorators) + + def visit_Class(self, node): + #self.name_types.append(_ast.Load) + bases = [self.visit(b) for b in node.bases] + #self.name_types.pop() + body = self.visit(node.code) + if node.doc: + body = [self._new(_ast.Expr, self._new(_ast.Str, node.doc))] + body + return self._new(_ast.ClassDef, node.name, bases, body) + + def visit_Return(self, node): + return self._new(_ast.Return, self.visit(node.value)) + + def visit_Assign(self, node): + #self.name_types.append(_ast.Store) + targets = [self.visit(t) for t in node.nodes] + #self.name_types.pop() + return self._new(_ast.Assign, targets, self.visit(node.expr)) + + aug_operators = { + '+=': _ast.Add, + '/=': _ast.Div, + '//=': _ast.FloorDiv, + '<<=': _ast.LShift, + '%=': _ast.Mod, + '*=': _ast.Mult, + '**=': _ast.Pow, + '>>=': _ast.RShift, + '-=': _ast.Sub, + } + + def visit_AugAssign(self, node): + target = self.visit(node.node) + + # Because it's AugAssign target can't be list nor tuple + # so we only have to change context of one node + target.ctx = _ast.Store() + op = self.aug_operators[node.op]() + return self._new(_ast.AugAssign, target, op, self.visit(node.expr)) + + def _visit_Print(nl): + def _visit(self, node): + values = [self.visit(v) for v in node.nodes] + return self._new(_ast.Print, self.visit(node.dest), values, nl) + return _visit + + visit_Print = _visit_Print(False) + visit_Printnl = _visit_Print(True) + del _visit_Print + + def visit_For(self, node): + return self._new(_ast.For, self.visit(node.assign), self.visit(node.list), + self.visit(node.body), self.visit(node.else_)) + + def visit_While(self, node): + return self._new(_ast.While, self.visit(node.test), self.visit(node.body), + self.visit(node.else_)) + + def visit_If(self, node): + def _level(tests, else_): + test = self.visit(tests[0][0]) + body = self.visit(tests[0][1]) + if len(tests) == 1: + orelse = self.visit(else_) + else: + orelse = [_level(tests[1:], else_)] + return self._new(_ast.If, test, body, orelse) + return _level(node.tests, node.else_) + + def visit_With(self, node): + return self._new(_ast.With, self.visit(node.expr), + self.visit(node.vars), self.visit(node.body)) + + def visit_Raise(self, node): + return self._new(_ast.Raise, self.visit(node.expr1), + self.visit(node.expr2), self.visit(node.expr3)) + + def visit_TryExcept(self, node): + handlers = [] + for type, name, body in node.handlers: + handlers.append(self._new(_ast.excepthandler, self.visit(type), + self.visit(name), self.visit(body))) + return self._new(_ast.TryExcept, self.visit(node.body), + handlers, self.visit(node.else_)) + + def visit_TryFinally(self, node): + return self._new(_ast.TryFinally, self.visit(node.body), + self.visit(node.final)) + + def visit_Assert(self, node): + return self._new(_ast.Assert, self.visit(node.test), self.visit(node.fail)) + + def visit_Import(self, node): + names = [self._new(_ast.alias, n[0], n[1]) for n in node.names] + return self._new(_ast.Import, names) + + def visit_From(self, node): + names = [self._new(_ast.alias, n[0], n[1]) for n in node.names] + return self._new(_ast.ImportFrom, node.modname, names, 0) + + def visit_Exec(self, node): + return self._new(_ast.Exec, self.visit(node.expr), + self.visit(node.locals), self.visit(node.globals)) + + def visit_Global(self, node): + return self._new(_ast.Global, node.names[:]) + + def visit_Discard(self, node): + return self._new(_ast.Expr, self.visit(node.expr)) + + def _map_class(to): + def _visit(self, node): + return self._new(to) + return _visit + + visit_Pass = _map_class(_ast.Pass) + visit_Break = _map_class(_ast.Break) + visit_Continue = _map_class(_ast.Continue) + + def _visit_BinOperator(opcls): + def _visit(self, node): + return self._new(_ast.BinOp, self.visit(node.left), + opcls(), self.visit(node.right)) + return _visit + visit_Add = _visit_BinOperator(_ast.Add) + visit_Div = _visit_BinOperator(_ast.Div) + visit_FloorDiv = _visit_BinOperator(_ast.FloorDiv) + visit_LeftShift = _visit_BinOperator(_ast.LShift) + visit_Mod = _visit_BinOperator(_ast.Mod) + visit_Mul = _visit_BinOperator(_ast.Mult) + visit_Power = _visit_BinOperator(_ast.Pow) + visit_RightShift = _visit_BinOperator(_ast.RShift) + visit_Sub = _visit_BinOperator(_ast.Sub) + del _visit_BinOperator + + def _visit_BitOperator(opcls): + def _visit(self, node): + def _make(nodes): + if len(nodes) == 1: + return self.visit(nodes[0]) + left = _make(nodes[:-1]) + right = self.visit(nodes[-1]) + return self._new(_ast.BinOp, left, opcls(), right) + return _make(node.nodes) + return _visit + visit_Bitand = _visit_BitOperator(_ast.BitAnd) + visit_Bitor = _visit_BitOperator(_ast.BitOr) + visit_Bitxor = _visit_BitOperator(_ast.BitXor) + del _visit_BitOperator + + def _visit_UnaryOperator(opcls): + def _visit(self, node): + return self._new(_ast.UnaryOp, opcls(), self.visit(node.expr)) + return _visit + + visit_Invert = _visit_UnaryOperator(_ast.Invert) + visit_Not = _visit_UnaryOperator(_ast.Not) + visit_UnaryAdd = _visit_UnaryOperator(_ast.UAdd) + visit_UnarySub = _visit_UnaryOperator(_ast.USub) + del _visit_UnaryOperator + + def _visit_BoolOperator(opcls): + def _visit(self, node): + values = [self.visit(n) for n in node.nodes] + return self._new(_ast.BoolOp, opcls(), values) + return _visit + visit_And = _visit_BoolOperator(_ast.And) + visit_Or = _visit_BoolOperator(_ast.Or) + del _visit_BoolOperator + + cmp_operators = { + '==': _ast.Eq, + '!=': _ast.NotEq, + '<': _ast.Lt, + '<=': _ast.LtE, + '>': _ast.Gt, + '>=': _ast.GtE, + 'is': _ast.Is, + 'is not': _ast.IsNot, + 'in': _ast.In, + 'not in': _ast.NotIn, + } + + def visit_Compare(self, node): + left = self.visit(node.expr) + ops = [] + comparators = [] + for optype, expr in node.ops: + ops.append(self.cmp_operators[optype]()) + comparators.append(self.visit(expr)) + return self._new(_ast.Compare, left, ops, comparators) + + def visit_Lambda(self, node): + args = self._extract_args(node) + body = self.visit(node.code) + return self._new(_ast.Lambda, args, body) + + def visit_IfExp(self, node): + return self._new(_ast.IfExp, self.visit(node.test), self.visit(node.then), + self.visit(node.else_)) + + def visit_Dict(self, node): + keys = [self.visit(x[0]) for x in node.items] + values = [self.visit(x[1]) for x in node.items] + return self._new(_ast.Dict, keys, values) + + def visit_ListComp(self, node): + generators = [self.visit(q) for q in node.quals] + return self._new(_ast.ListComp, self.visit(node.expr), generators) + + def visit_GenExprInner(self, node): + generators = [self.visit(q) for q in node.quals] + return self._new(_ast.GeneratorExp, self.visit(node.expr), generators) + + def visit_GenExpr(self, node): + return self.visit(node.code) + + def visit_GenExprFor(self, node): + ifs = [self.visit(i) for i in node.ifs] + return self._new(_ast.comprehension, self.visit(node.assign), + self.visit(node.iter), ifs) + + def visit_ListCompFor(self, node): + ifs = [self.visit(i) for i in node.ifs] + return self._new(_ast.comprehension, self.visit(node.assign), + self.visit(node.list), ifs) + + def visit_GenExprIf(self, node): + return self.visit(node.test) + visit_ListCompIf = visit_GenExprIf + + def visit_Yield(self, node): + return self._new(_ast.Yield, self.visit(node.value)) + + def visit_CallFunc(self, node): + args = [] + keywords = [] + for arg in node.args: + if isinstance(arg, compiler.ast.Keyword): + keywords.append(self._new(_ast.keyword, arg.name, + self.visit(arg.expr))) + else: + args.append(self.visit(arg)) + return self._new(_ast.Call, self.visit(node.node), args, keywords, + self.visit(node.star_args), self.visit(node.dstar_args)) + + def visit_Backquote(self, node): + return self._new(_ast.Repr, self.visit(node.expr)) + + def visit_Const(self, node): + if node.value is None: # appears in slices + return None + elif isinstance(node.value, (str, unicode,)): + return self._new(_ast.Str, node.value) + else: + return self._new(_ast.Num, node.value) + + def visit_Name(self, node): + return self._new(_ast.Name, node.name, _ast.Load()) + + def visit_Getattr(self, node): + return self._new(_ast.Attribute, self.visit(node.expr), node.attrname, + _ast.Load()) + + def visit_Tuple(self, node): + nodes = [self.visit(n) for n in node.nodes] + return self._new(_ast.Tuple, nodes, _ast.Load()) + + def visit_List(self, node): + nodes = [self.visit(n) for n in node.nodes] + return self._new(_ast.List, nodes, _ast.Load()) + + def get_ctx(self, flags): + if flags == 'OP_DELETE': + return _ast.Del() + elif flags == 'OP_APPLY': + return _ast.Load() + elif flags == 'OP_ASSIGN': + return _ast.Store() + else: + # FIXME Exception here + assert False, repr(flags) + + def visit_AssName(self, node): + self.out_flags = node.flags + ctx = self.get_ctx(node.flags) + return self._new(_ast.Name, node.name, ctx) + + def visit_AssAttr(self, node): + self.out_flags = node.flags + ctx = self.get_ctx(node.flags) + return self._new(_ast.Attribute, self.visit(node.expr), + node.attrname, ctx) + + def _visit_AssCollection(cls): + def _visit(self, node): + flags = None + elts = [] + for n in node.nodes: + elts.append(self.visit(n)) + if flags is None: + flags = self.out_flags + else: + assert flags == self.out_flags + self.out_flags = flags + ctx = self.get_ctx(flags) + return self._new(cls, elts, ctx) + return _visit + + visit_AssList = _visit_AssCollection(_ast.List) + visit_AssTuple = _visit_AssCollection(_ast.Tuple) + del _visit_AssCollection + + def visit_Slice(self, node): + lower = self.visit(node.lower) + upper = self.visit(node.upper) + ctx = self.get_ctx(node.flags) + self.out_flags = node.flags + return self._new(_ast.Subscript, self.visit(node.expr), + self._new(_ast.Slice, lower, upper, None), ctx) + + def visit_Subscript(self, node): + ctx = self.get_ctx(node.flags) + subs = [self.visit(s) for s in node.subs] + + advanced = (_ast.Slice, _ast.Ellipsis) + slices = [] + nonindex = False + for sub in subs: + if isinstance(sub, advanced): + nonindex = True + slices.append(sub) + else: + slices.append(self._new(_ast.Index, sub)) + if len(slices) == 1: + slice = slices[0] + elif nonindex: + slice = self._new(_ast.ExtSlice, slices) + else: + slice = self._new(_ast.Tuple, slices, _ast.Load()) + + self.out_flags = node.flags + return self._new(_ast.Subscript, self.visit(node.expr), slice, ctx) + + def visit_Sliceobj(self, node): + a = node.nodes + [None]*(3 - len(node.nodes)) + a = map(self.visit, a) + return self._new(_ast.Slice, a[0], a[1], a[2]) + + def visit_Ellipsis(self, node): + return self._new(_ast.Ellipsis) + + def visit_Stmt(self, node): + def _check_del(n): + # del x is just AssName('x', 'OP_DELETE') + # we want to transform it to Delete([Name('x', Del())]) + dcls = (_ast.Name, _ast.List, _ast.Subscript, _ast.Attribute) + if isinstance(n, dcls) and isinstance(n.ctx, _ast.Del): + return self._new(_ast.Delete, [n]) + elif isinstance(n, _ast.Tuple) and isinstance(n.ctx, _ast.Del): + # unpack last tuple to avoid making del (x, y, z,); + # out of del x, y, z; (there's no difference between + # this two in compiler.ast) + return self._new(_ast.Delete, n.elts) + else: + return n + def _keep(n): + if isinstance(n, _ast.Expr) and n.value is None: + return False + else: + return True + statements = [_check_del(self.visit(n)) for n in node.nodes] + return filter(_keep, statements) + + +def parse(source, mode): + node = compiler.parse(source, mode) + return ASTUpgrader().visit(node)
new file mode 100644 --- /dev/null +++ b/genshi/template/astgae.py @@ -0,0 +1,104 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2008 Edgewall Software +# All rights reserved. +# +# This software is licensed as described in the file COPYING, which +# you should have received as part of this distribution. The terms +# are also available at http://genshi.edgewall.org/wiki/License. +# +# This software consists of voluntary contributions made by many +# individuals. For the exact contribution history, see the revision +# history and logs, available at http://genshi.edgewall.org/log/. + +"""Support for using the Python AST on Google App Engine.""" + +__all__ = ['restore'] +__docformat__ = 'restructuredtext en' + + +def restore(_ast): + """Gross hack to restore the required classes to the _ast module if it + appears to be missing them. Mostly lifted from Mako. + """ + _ast.PyCF_ONLY_AST = 2 << 9 + + e = compile('True', '<string>', 'eval', _ast.PyCF_ONLY_AST) + _ast.Expression = type(e) + for cls in _ast.Expression.__mro__: + if cls.__name__ == 'AST': + _ast.AST = cls + + m = compile("""\ +foo() +bar = 'fish' +baz += bar +1 + 2 - 3 * 4 / 5 ** 6 +6 // 7 % 8 << 9 >> 10 +11 & 12 ^ 13 | 14 +15 and 16 or 17 +-baz + (not +18) - ~17 +baz and 'foo' or 'bar' +(fish is baz == baz) is not baz != fish +fish > baz < fish >= baz <= fish +fish in baz not in (1, 2, 3) +baz[1, 1:2, ...] +""", '<string>', 'exec', _ast.PyCF_ONLY_AST) + + _ast.Module = type(m) + + _ast.Expr = type(m.body[0]) + _ast.Call = type(m.body[0].value) + + _ast.Assign = type(m.body[1]) + _ast.Name = type(m.body[1].targets[0]) + _ast.Store = type(m.body[1].targets[0].ctx) + _ast.Str = type(m.body[1].value) + + _ast.AugAssign = type(m.body[2]) + _ast.Load = type(m.body[2].value.ctx) + + _ast.Sub = type(m.body[3].value.op) + _ast.Add = type(m.body[3].value.left.op) + _ast.Div = type(m.body[3].value.right.op) + _ast.Mult = type(m.body[3].value.right.left.op) + _ast.Pow = type(m.body[3].value.right.right.op) + + _ast.RShift = type(m.body[4].value.op) + _ast.LShift = type(m.body[4].value.left.op) + _ast.Mod = type(m.body[4].value.left.left.op) + _ast.FloorDiv = type(m.body[4].value.left.left.left.op) + + _ast.BitOr = type(m.body[5].value.op) + _ast.BitXor = type(m.body[5].value.left.op) + _ast.BitAnd = type(m.body[5].value.left.left.op) + + _ast.Or = type(m.body[6].value.op) + _ast.And = type(m.body[6].value.values[0].op) + + _ast.Invert = type(m.body[7].value.right.op) + _ast.Not = type(m.body[7].value.left.right.op) + _ast.UAdd = type(m.body[7].value.left.right.operand.op) + _ast.USub = type(m.body[7].value.left.left.op) + + _ast.Or = type(m.body[8].value.op) + _ast.And = type(m.body[8].value.values[0].op) + + _ast.IsNot = type(m.body[9].value.ops[0]) + _ast.NotEq = type(m.body[9].value.ops[1]) + _ast.Is = type(m.body[9].value.left.ops[0]) + _ast.Eq = type(m.body[9].value.left.ops[1]) + + _ast.Gt = type(m.body[10].value.ops[0]) + _ast.Lt = type(m.body[10].value.ops[1]) + _ast.GtE = type(m.body[10].value.ops[2]) + _ast.LtE = type(m.body[10].value.ops[3]) + + _ast.In = type(m.body[11].value.ops[0]) + _ast.NotIn = type(m.body[11].value.ops[1]) + _ast.Tuple = type(m.body[11].value.comparators[1]) + + _ast.ExtSlice = type(m.body[12].value.slice) + _ast.Index = type(m.body[12].value.slice.dims[0]) + _ast.Slice = type(m.body[12].value.slice.dims[1]) + _ast.Ellipsis = type(m.body[12].value.slice.dims[2])
new file mode 100644 --- /dev/null +++ b/genshi/template/astutil.py @@ -0,0 +1,781 @@ +# -*- coding: utf-8 -*- +# +# Copyright (C) 2008 Edgewall Software +# All rights reserved. +# +# This software is licensed as described in the file COPYING, which +# you should have received as part of this distribution. The terms +# are also available at http://genshi.edgewall.org/wiki/License. +# +# This software consists of voluntary contributions made by many +# individuals. For the exact contribution history, see the revision +# history and logs, available at http://genshi.edgewall.org/log/. + +"""Support classes for generating code from abstract syntax trees.""" + +try: + import _ast +except ImportError: + from genshi.template.ast24 import _ast, parse +else: + if not hasattr(_ast, 'AST'): + from genshi.template.astgae import restore + restore(_ast) + def parse(source, mode): + return compile(source, '', mode, _ast.PyCF_ONLY_AST) + + +__docformat__ = 'restructuredtext en' + + +class ASTCodeGenerator(object): + """General purpose base class for AST transformations. + + Every visitor method can be overridden to return an AST node that has been + altered or replaced in some way. + """ + def __init__(self, tree): + self.lines_info = [] + self.line_info = None + self.code = '' + self.line = None + self.last = None + self.indent = 0 + self.blame_stack = [] + self.visit(tree) + if self.line.strip(): + self.code += self.line + '\n' + self.lines_info.append(self.line_info) + self.line = None + self.line_info = None + + def _change_indent(self, delta): + self.indent += delta + + def _new_line(self): + if self.line is not None: + self.code += self.line + '\n' + self.lines_info.append(self.line_info) + self.line = ' '*4*self.indent + if len(self.blame_stack) == 0: + self.line_info = [] + self.last = None + else: + self.line_info = [(0, self.blame_stack[-1],)] + self.last = self.blame_stack[-1] + + def _write(self, s): + if len(s) == 0: + return + if len(self.blame_stack) == 0: + if self.last is not None: + self.last = None + self.line_info.append((len(self.line), self.last)) + else: + if self.last != self.blame_stack[-1]: + self.last = self.blame_stack[-1] + self.line_info.append((len(self.line), self.last)) + self.line += s + + def visit(self, node): + if node is None: + return None + if type(node) is tuple: + return tuple([self.visit(n) for n in node]) + try: + self.blame_stack.append((node.lineno, node.col_offset,)) + info = True + except AttributeError: + info = False + visitor = getattr(self, 'visit_%s' % node.__class__.__name__, None) + if visitor is None: + raise Exception('Unhandled node type %r' % type(node)) + ret = visitor(node) + if info: + self.blame_stack.pop() + return ret + + def visit_Module(self, node): + for n in node.body: + self.visit(n) + visit_Interactive = visit_Module + visit_Suite = visit_Module + + def visit_Expression(self, node): + self._new_line() + return self.visit(node.body) + + # arguments = (expr* args, identifier? vararg, + # identifier? kwarg, expr* defaults) + def visit_arguments(self, node): + first = True + no_default_count = len(node.args) - len(node.defaults) + for i, arg in enumerate(node.args): + if not first: + self._write(', ') + else: + first = False + self.visit(arg) + if i >= no_default_count: + self._write('=') + self.visit(node.defaults[i - no_default_count]) + if getattr(node, 'vararg', None): + if not first: + self._write(', ') + else: + first = False + self._write('*' + node.vararg) + if getattr(node, 'kwarg', None): + if not first: + self._write(', ') + else: + first = False + self._write('**' + node.kwarg) + + # FunctionDef(identifier name, arguments args, + # stmt* body, expr* decorators) + def visit_FunctionDef(self, node): + for decorator in getattr(node, 'decorators', ()): + self._new_line() + self._write('@') + self.visit(decorator) + self._new_line() + self._write('def ' + node.name + '(') + self.visit(node.args) + self._write('):') + self._change_indent(1) + for statement in node.body: + self.visit(statement) + self._change_indent(-1) + + # ClassDef(identifier name, expr* bases, stmt* body) + def visit_ClassDef(self, node): + self._new_line() + self._write('class ' + node.name) + if node.bases: + self._write('(') + self.visit(node.bases[0]) + for base in node.bases[1:]: + self._write(', ') + self.visit(base) + self._write(')') + self._write(':') + self._change_indent(1) + for statement in node.body: + self.visit(statement) + self._change_indent(-1) + + # Return(expr? value) + def visit_Return(self, node): + self._new_line() + self._write('return') + if getattr(node, 'value', None): + self._write(' ') + self.visit(node.value) + + # Delete(expr* targets) + def visit_Delete(self, node): + self._new_line() + self._write('del ') + self.visit(node.targets[0]) + for target in node.targets[1:]: + self._write(', ') + self.visit(target) + + # Assign(expr* targets, expr value) + def visit_Assign(self, node): + self._new_line() + for target in node.targets: + self.visit(target) + self._write(' = ') + self.visit(node.value) + + # AugAssign(expr target, operator op, expr value) + def visit_AugAssign(self, node): + self._new_line() + self.visit(node.target) + self._write(' ' + self.binary_operators[node.op.__class__] + '= ') + self.visit(node.value) + + # Print(expr? dest, expr* values, bool nl) + def visit_Print(self, node): + self._new_line() + self._write('print') + if getattr(node, 'dest', None): + self._write(' >> ') + self.visit(node.dest) + if getattr(node, 'values', None): + self._write(', ') + else: + self._write(' ') + if getattr(node, 'values', None): + self.visit(node.values[0]) + for value in node.values[1:]: + self._write(', ') + self.visit(value) + if not node.nl: + self._write(',') + + # For(expr target, expr iter, stmt* body, stmt* orelse) + def visit_For(self, node): + self._new_line() + self._write('for ') + self.visit(node.target) + self._write(' in ') + self.visit(node.iter) + self._write(':') + self._change_indent(1) + for statement in node.body: + self.visit(statement) + self._change_indent(-1) + if getattr(node, 'orelse', None): + self._new_line() + self._write('else:') + self._change_indent(1) + for statement in node.orelse: + self.visit(statement) + self._change_indent(-1) + + # While(expr test, stmt* body, stmt* orelse) + def visit_While(self, node): + self._new_line() + self._write('while ') + self.visit(node.test) + self._write(':') + self._change_indent(1) + for statement in node.body: + self.visit(statement) + self._change_indent(-1) + if getattr(node, 'orelse', None): + self._new_line() + self._write('else:') + self._change_indent(1) + for statement in node.orelse: + self.visit(statement) + self._change_indent(-1) + + # If(expr test, stmt* body, stmt* orelse) + def visit_If(self, node): + self._new_line() + self._write('if ') + self.visit(node.test) + self._write(':') + self._change_indent(1) + for statement in node.body: + self.visit(statement) + self._change_indent(-1) + if getattr(node, 'orelse', None): + self._new_line() + self._write('else:') + self._change_indent(1) + for statement in node.orelse: + self.visit(statement) + self._change_indent(-1) + + # With(expr context_expr, expr? optional_vars, stmt* body) + def visit_With(self, node): + self._new_line() + self._write('with ') + self.visit(node.context_expr) + if getattr(node, 'optional_vars', None): + self._write(' as ') + self.visit(node.optional_vars) + self._write(':') + self._change_indent(1) + for statement in node.body: + self.visit(statement) + self._change_indent(-1) + + + # Raise(expr? type, expr? inst, expr? tback) + def visit_Raise(self, node): + self._new_line() + self._write('raise') + if not node.type: + return + self._write(' ') + self.visit(node.type) + if not node.inst: + return + self._write(', ') + self.visit(node.inst) + if not node.tback: + return + self._write(', ') + self.visit(node.tback) + + # TryExcept(stmt* body, excepthandler* handlers, stmt* orelse) + def visit_TryExcept(self, node): + self._new_line() + self._write('try:') + self._change_indent(1) + for statement in node.body: + self.visit(statement) + self._change_indent(-1) + if getattr(node, 'handlers', None): + for handler in node.handlers: + self.visit(handler) + self._new_line() + if getattr(node, 'orelse', None): + self._write('else:') + self._change_indent(1) + for statement in node.orelse: + self.visit(statement) + self._change_indent(-1) + + # excepthandler = (expr? type, expr? name, stmt* body) + def visit_ExceptHandler(self, node): + self._new_line() + self._write('except') + if getattr(node, 'type', None): + self._write(' ') + self.visit(node.type) + if getattr(node, 'name', None): + self._write(', ') + self.visit(node.name) + self._write(':') + self._change_indent(1) + for statement in node.body: + self.visit(statement) + self._change_indent(-1) + visit_excepthandler = visit_ExceptHandler + + # TryFinally(stmt* body, stmt* finalbody) + def visit_TryFinally(self, node): + self._new_line() + self._write('try:') + self._change_indent(1) + for statement in node.body: + self.visit(statement) + self._change_indent(-1) + + if getattr(node, 'finalbody', None): + self._new_line() + self._write('finally:') + self._change_indent(1) + for statement in node.finalbody: + self.visit(statement) + self._change_indent(-1) + + # Assert(expr test, expr? msg) + def visit_Assert(self, node): + self._new_line() + self._write('assert ') + self.visit(node.test) + if getattr(node, 'msg', None): + self._write(', ') + self.visit(node.msg) + + def visit_alias(self, node): + self._write(node.name) + if getattr(node, 'asname', None): + self._write(' as ') + self._write(node.asname) + + # Import(alias* names) + def visit_Import(self, node): + self._new_line() + self._write('import ') + self.visit(node.names[0]) + for name in node.names[1:]: + self._write(', ') + self.visit(name) + + # ImportFrom(identifier module, alias* names, int? level) + def visit_ImportFrom(self, node): + self._new_line() + self._write('from ') + if node.level: + self._write('.' * node.level) + self._write(node.module) + self._write(' import ') + self.visit(node.names[0]) + for name in node.names[1:]: + self._write(', ') + self.visit(name) + + # Exec(expr body, expr? globals, expr? locals) + def visit_Exec(self, node): + self._new_line() + self._write('exec ') + self.visit(node.body) + if not node.globals: + return + self._write(', ') + self.visit(node.globals) + if not node.locals: + return + self._write(', ') + self.visit(node.locals) + + # Global(identifier* names) + def visit_Global(self, node): + self._new_line() + self._write('global ') + self.visit(node.names[0]) + for name in node.names[1:]: + self._write(', ') + self.visit(name) + + # Expr(expr value) + def visit_Expr(self, node): + self._new_line() + self.visit(node.value) + + # Pass + def visit_Pass(self, node): + self._new_line() + self._write('pass') + + # Break + def visit_Break(self, node): + self._new_line() + self._write('break') + + # Continue + def visit_Continue(self, node): + self._new_line() + self._write('continue') + + ### EXPRESSIONS + def with_parens(f): + def _f(self, node): + self._write('(') + f(self, node) + self._write(')') + return _f + + bool_operators = {_ast.And: 'and', _ast.Or: 'or'} + + # BoolOp(boolop op, expr* values) + @with_parens + def visit_BoolOp(self, node): + joiner = ' ' + self.bool_operators[node.op.__class__] + ' ' + self.visit(node.values[0]) + for value in node.values[1:]: + self._write(joiner) + self.visit(value) + + binary_operators = { + _ast.Add: '+', + _ast.Sub: '-', + _ast.Mult: '*', + _ast.Div: '/', + _ast.Mod: '%', + _ast.Pow: '**', + _ast.LShift: '<<', + _ast.RShift: '>>', + _ast.BitOr: '|', + _ast.BitXor: '^', + _ast.BitAnd: '&', + _ast.FloorDiv: '//' + } + + # BinOp(expr left, operator op, expr right) + @with_parens + def visit_BinOp(self, node): + self.visit(node.left) + self._write(' ' + self.binary_operators[node.op.__class__] + ' ') + self.visit(node.right) + + unary_operators = { + _ast.Invert: '~', + _ast.Not: 'not', + _ast.UAdd: '+', + _ast.USub: '-', + } + + # UnaryOp(unaryop op, expr operand) + def visit_UnaryOp(self, node): + self._write(self.unary_operators[node.op.__class__] + ' ') + self.visit(node.operand) + + # Lambda(arguments args, expr body) + @with_parens + def visit_Lambda(self, node): + self._write('lambda ') + self.visit(node.args) + self._write(': ') + self.visit(node.body) + + # IfExp(expr test, expr body, expr orelse) + @with_parens + def visit_IfExp(self, node): + self.visit(node.body) + self._write(' if ') + self.visit(node.test) + self._write(' else ') + self.visit(node.orelse) + + # Dict(expr* keys, expr* values) + def visit_Dict(self, node): + self._write('{') + for key, value in zip(node.keys, node.values): + self.visit(key) + self._write(': ') + self.visit(value) + self._write(', ') + self._write('}') + + # ListComp(expr elt, comprehension* generators) + def visit_ListComp(self, node): + self._write('[') + self.visit(node.elt) + for generator in node.generators: + # comprehension = (expr target, expr iter, expr* ifs) + self._write(' for ') + self.visit(generator.target) + self._write(' in ') + self.visit(generator.iter) + for ifexpr in generator.ifs: + self._write(' if ') + self.visit(ifexpr) + self._write(']') + + # GeneratorExp(expr elt, comprehension* generators) + def visit_GeneratorExp(self, node): + self._write('(') + self.visit(node.elt) + for generator in node.generators: + # comprehension = (expr target, expr iter, expr* ifs) + self._write(' for ') + self.visit(generator.target) + self._write(' in ') + self.visit(generator.iter) + for ifexpr in generator.ifs: + self._write(' if ') + self.visit(ifexpr) + self._write(')') + + # Yield(expr? value) + def visit_Yield(self, node): + self._write('yield') + if getattr(node, 'value', None): + self._write(' ') + self.visit(node.value) + + comparision_operators = { + _ast.Eq: '==', + _ast.NotEq: '!=', + _ast.Lt: '<', + _ast.LtE: '<=', + _ast.Gt: '>', + _ast.GtE: '>=', + _ast.Is: 'is', + _ast.IsNot: 'is not', + _ast.In: 'in', + _ast.NotIn: 'not in', + } + + # Compare(expr left, cmpop* ops, expr* comparators) + @with_parens + def visit_Compare(self, node): + self.visit(node.left) + for op, comparator in zip(node.ops, node.comparators): + self._write(' ' + self.comparision_operators[op.__class__] + ' ') + self.visit(comparator) + + # Call(expr func, expr* args, keyword* keywords, + # expr? starargs, expr? kwargs) + def visit_Call(self, node): + self.visit(node.func) + self._write('(') + first = True + for arg in node.args: + if not first: + self._write(', ') + first = False + self.visit(arg) + + for keyword in node.keywords: + if not first: + self._write(', ') + first = False + # keyword = (identifier arg, expr value) + self._write(keyword.arg) + self._write('=') + self.visit(keyword.value) + if getattr(node, 'starargs', None): + if not first: + self._write(', ') + first = False + self._write('*') + self.visit(node.starargs) + + if getattr(node, 'kwargs', None): + if not first: + self._write(', ') + first = False + self._write('**') + self.visit(node.kwargs) + self._write(')') + + # Repr(expr value) + def visit_Repr(self, node): + self._write('`') + self.visit(node.value) + self._write('`') + + # Num(object n) + def visit_Num(self, node): + self._write(repr(node.n)) + + # Str(string s) + def visit_Str(self, node): + self._write(repr(node.s)) + + # Attribute(expr value, identifier attr, expr_context ctx) + def visit_Attribute(self, node): + self.visit(node.value) + self._write('.') + self._write(node.attr) + + # Subscript(expr value, slice slice, expr_context ctx) + def visit_Subscript(self, node): + self.visit(node.value) + self._write('[') + def _process_slice(node): + if isinstance(node, _ast.Ellipsis): + self._write('...') + elif isinstance(node, _ast.Slice): + if getattr(node, 'lower', 'None'): + self.visit(node.lower) + self._write(':') + if getattr(node, 'upper', None): + self.visit(node.upper) + if getattr(node, 'step', None): + self._write(':') + self.visit(node.step) + elif isinstance(node, _ast.Index): + self.visit(node.value) + elif isinstance(node, _ast.ExtSlice): + self.visit(node.dims[0]) + for dim in node.dims[1:]: + self._write(', ') + self.visit(dim) + else: + raise NotImplemented, 'Slice type not implemented' + _process_slice(node.slice) + self._write(']') + + # Name(identifier id, expr_context ctx) + def visit_Name(self, node): + self._write(node.id) + + # List(expr* elts, expr_context ctx) + def visit_List(self, node): + self._write('[') + for elt in node.elts: + self.visit(elt) + self._write(', ') + self._write(']') + + # Tuple(expr *elts, expr_context ctx) + def visit_Tuple(self, node): + self._write('(') + for elt in node.elts: + self.visit(elt) + self._write(', ') + self._write(')') + + +class ASTTransformer(object): + """General purpose base class for AST transformations. + + Every visitor method can be overridden to return an AST node that has been + altered or replaced in some way. + """ + + def visit(self, node): + if node is None: + return None + if type(node) is tuple: + return tuple([self.visit(n) for n in node]) + visitor = getattr(self, 'visit_%s' % node.__class__.__name__, None) + if visitor is None: + return node + return visitor(node) + + def _clone(self, node): + clone = node.__class__() + for name in getattr(clone, '_attributes', ()): + try: + setattr(clone, 'name', getattr(node, name)) + except AttributeError: + pass + for name in clone._fields: + try: + value = getattr(node, name) + except AttributeError: + pass + else: + if value is None: + pass + elif isinstance(value, list): + value = [self.visit(x) for x in value] + elif isinstance(value, tuple): + value = tuple(self.visit(x) for x in value) + else: + value = self.visit(value) + setattr(clone, name, value) + return clone + + visit_Module = _clone + visit_Interactive = _clone + visit_Expression = _clone + visit_Suite = _clone + + visit_FunctionDef = _clone + visit_ClassDef = _clone + visit_Return = _clone + visit_Delete = _clone + visit_Assign = _clone + visit_AugAssign = _clone + visit_Print = _clone + visit_For = _clone + visit_While = _clone + visit_If = _clone + visit_With = _clone + visit_Raise = _clone + visit_TryExcept = _clone + visit_TryFinally = _clone + visit_Assert = _clone + + visit_Import = _clone + visit_ImportFrom = _clone + visit_Exec = _clone + visit_Global = _clone + visit_Expr = _clone + # Pass, Break, Continue don't need to be copied + + visit_BoolOp = _clone + visit_BinOp = _clone + visit_UnaryOp = _clone + visit_Lambda = _clone + visit_IfExp = _clone + visit_Dict = _clone + visit_ListComp = _clone + visit_GeneratorExp = _clone + visit_Yield = _clone + visit_Compare = _clone + visit_Call = _clone + visit_Repr = _clone + # Num, Str don't need to be copied + + visit_Attribute = _clone + visit_Subscript = _clone + visit_Name = _clone + visit_List = _clone + visit_Tuple = _clone + + visit_comprehension = _clone + visit_excepthandler = _clone + visit_arguments = _clone + visit_keyword = _clone + visit_alias = _clone + + visit_Slice = _clone + visit_ExtSlice = _clone + visit_Index = _clone + + del _clone
--- a/genshi/template/base.py +++ b/genshi/template/base.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -21,19 +21,20 @@ def popleft(self): return self.pop(0) import os from StringIO import StringIO +import sys from genshi.core import Attrs, Stream, StreamEventKind, START, TEXT, _ensure from genshi.input import ParseError -__all__ = ['Context', 'Template', 'TemplateError', 'TemplateRuntimeError', - 'TemplateSyntaxError', 'BadDirectiveError'] +__all__ = ['Context', 'DirectiveFactory', 'Template', 'TemplateError', + 'TemplateRuntimeError', 'TemplateSyntaxError', 'BadDirectiveError'] __docformat__ = 'restructuredtext en' class TemplateError(Exception): """Base exception class for errors related to template processing.""" - def __init__(self, message, filename='<string>', lineno=-1, offset=-1): + def __init__(self, message, filename=None, lineno=-1, offset=-1): """Create the exception. :param message: the error message @@ -42,6 +43,8 @@ occurred :param offset: the column number at which the error occurred """ + if filename is None: + filename = '<string>' self.msg = message #: the error message string if filename != '<string>' or lineno >= 0: message = '%s (%s, line %d)' % (self.msg, filename, lineno) @@ -56,7 +59,7 @@ error, or the template is not well-formed. """ - def __init__(self, message, filename='<string>', lineno=-1, offset=-1): + def __init__(self, message, filename=None, lineno=-1, offset=-1): """Create the exception :param message: the error message @@ -78,7 +81,7 @@ with a local name that doesn't match any registered directive. """ - def __init__(self, name, filename='<string>', lineno=-1): + def __init__(self, name, filename=None, lineno=-1): """Create the exception :param name: the name of the directive @@ -129,6 +132,7 @@ self.pop = self.frames.popleft self.push = self.frames.appendleft self._match_templates = [] + self._choice_stack = [] # Helper functions for use in expressions def defined(name): @@ -151,6 +155,7 @@ :param key: the name of the variable """ return self._find(key)[1] is not None + has_key = __contains__ def __delitem__(self, key): """Remove a variable from all scopes. @@ -234,6 +239,10 @@ """ return [(key, self.get(key)) for key in self.keys()] + def update(self, mapping): + """Update the context from the mapping provided.""" + self.frames[0].update(mapping) + def push(self, data): """Push a new scope on the stack. @@ -244,21 +253,56 @@ """Pop the top-most scope from the stack.""" -def _apply_directives(stream, ctxt, directives): +def _apply_directives(stream, directives, ctxt, **vars): """Apply the given directives to the stream. :param stream: the stream the directives should be applied to + :param directives: the list of directives to apply :param ctxt: the `Context` - :param directives: the list of directives to apply + :param vars: additional variables that should be available when Python + code is executed :return: the stream with the given directives applied """ if directives: - stream = directives[0](iter(stream), ctxt, directives[1:]) + stream = directives[0](iter(stream), directives[1:], ctxt, **vars) return stream +def _eval_expr(expr, ctxt, **vars): + """Evaluate the given `Expression` object. + + :param expr: the expression to evaluate + :param ctxt: the `Context` + :param vars: additional variables that should be available to the + expression + :return: the result of the evaluation + """ + if vars: + ctxt.push(vars) + retval = expr.evaluate(ctxt) + if vars: + ctxt.pop() + return retval -class TemplateMeta(type): - """Meta class for templates.""" +def _exec_suite(suite, ctxt, **vars): + """Execute the given `Suite` object. + + :param suite: the code suite to execute + :param ctxt: the `Context` + :param vars: additional variables that should be available to the + code + """ + if vars: + ctxt.push(vars) + ctxt.push({}) + suite.execute(ctxt) + if vars: + top = ctxt.pop() + ctxt.pop() + ctxt.frames[0].update(top) + + +class DirectiveFactoryMeta(type): + """Meta class for directive factories.""" def __new__(cls, name, bases, d): if 'directives' in d: @@ -268,13 +312,47 @@ return type.__new__(cls, name, bases, d) -class Template(object): +class DirectiveFactory(object): + """Base for classes that provide a set of template directives. + + :since: version 0.6 + """ + __metaclass__ = DirectiveFactoryMeta + + directives = [] + """A list of `(name, cls)` tuples that define the set of directives + provided by this factory. + """ + + def compare_directives(self): + """Return a function that takes two directive classes and compares + them to determine their relative ordering. + """ + def _get_index(cls): + if cls in self._dir_order: + return self._dir_order.index(cls) + return 0 + return lambda a, b: cmp(_get_index(a[0]), _get_index(b[0])) + + def get_directive(self, name): + """Return the directive class for the given name. + + :param name: the directive name as used in the template + :return: the directive class + :see: `Directive` + """ + return self._dir_by_name.get(name) + + +class Template(DirectiveFactory): """Abstract template base class. This class implements most of the template processing model, but does not specify the syntax of templates. """ - __metaclass__ = TemplateMeta + + EXEC = StreamEventKind('EXEC') + """Stream event kind representing a Python code suite to execute.""" EXPR = StreamEventKind('EXPR') """Stream event kind representing a Python expression.""" @@ -287,49 +365,70 @@ directives should be applied. """ - def __init__(self, source, basedir=None, filename=None, loader=None, - encoding=None, lookup='lenient'): + serializer = None + _number_conv = unicode # function used to convert numbers to event data + + def __init__(self, source, filepath=None, filename=None, loader=None, + encoding=None, lookup='strict', allow_exec=True): """Initialize a template from either a string, a file-like object, or an already parsed markup stream. :param source: a string, file-like object, or markup stream to read the template from - :param basedir: the base directory containing the template file; when - loaded from a `TemplateLoader`, this will be the - directory on the template search path in which the - template was found - :param filename: the name of the template file, relative to the given - base directory + :param filepath: the absolute path to the template file + :param filename: the path to the template file relative to the search + path :param loader: the `TemplateLoader` to use for loading included templates :param encoding: the encoding of the `source` - :param lookup: the variable lookup mechanism; either "lenient" (the - default), "strict", or a custom lookup class + :param lookup: the variable lookup mechanism; either "strict" (the + default), "lenient", or a custom lookup class + :param allow_exec: whether Python code blocks in templates should be + allowed + + :note: Changed in 0.5: Added the `allow_exec` argument """ - self.basedir = basedir + self.filepath = filepath or filename self.filename = filename - if basedir and filename: - self.filepath = os.path.join(basedir, filename) - else: - self.filepath = filename self.loader = loader self.lookup = lookup + self.allow_exec = allow_exec + self._init_filters() + self._prepared = False if isinstance(source, basestring): source = StringIO(source) else: source = source try: - self.stream = list(self._prepare(self._parse(source, encoding))) + self._stream = self._parse(source, encoding) except ParseError, e: raise TemplateSyntaxError(e.msg, self.filepath, e.lineno, e.offset) - self.filters = [self._flatten, self._eval] - if loader: - self.filters.append(self._include) + + def __getstate__(self): + state = self.__dict__.copy() + state['filters'] = [] + return state + + def __setstate__(self, state): + self.__dict__ = state + self._init_filters() def __repr__(self): return '<%s "%s">' % (self.__class__.__name__, self.filename) + def _init_filters(self): + self.filters = [self._flatten] + if self.loader: + self.filters.append(self._include) + + def _get_stream(self): + if not self._prepared: + self._stream = list(self._prepare(self._stream)) + self._prepared = True + return self._stream + stream = property(_get_stream) + def _parse(self, source, encoding): """Parse the template. @@ -349,6 +448,8 @@ :param stream: the event stream of the template """ + from genshi.template.loader import TemplateNotFound + for kind, data, pos in stream: if kind is SUB: directives = [] @@ -366,9 +467,43 @@ yield event else: if kind is INCLUDE: - data = data[0], list(self._prepare(data[1])) + href, cls, fallback = data + if isinstance(href, basestring) and \ + not getattr(self.loader, 'auto_reload', True): + # If the path to the included template is static, and + # auto-reloading is disabled on the template loader, + # the template is inlined into the stream + try: + tmpl = self.loader.load(href, relative_to=pos[0], + cls=cls or self.__class__) + for event in tmpl.stream: + yield event + except TemplateNotFound: + if fallback is None: + raise + for event in self._prepare(fallback): + yield event + continue + elif fallback: + # Otherwise the include is performed at run time + data = href, cls, list(self._prepare(fallback)) + yield kind, data, pos + def compile(self): + """Compile the template to a Python module, and return the module + object. + """ + from imp import new_module + from genshi.template.inline import inline + + name = (self.filename or '_some_ident').replace('.', '_') + module = new_module(name) + source = u'\n'.join(list(inline(self))) + code = compile(source, self.filepath or '<string>', 'exec') + exec code in module.__dict__, module.__dict__ + return module + def generate(self, *args, **kwargs): """Apply the template to the given context data. @@ -382,25 +517,25 @@ :return: a markup event stream representing the result of applying the template to the context data. """ + vars = {} if args: assert len(args) == 1 ctxt = args[0] if ctxt is None: ctxt = Context(**kwargs) + else: + vars = kwargs assert isinstance(ctxt, Context) else: ctxt = Context(**kwargs) stream = self.stream for filter_ in self.filters: - stream = filter_(iter(stream), ctxt) - return Stream(stream) + stream = filter_(iter(stream), ctxt, **vars) + return Stream(stream, self.serializer) - def _eval(self, stream, ctxt): - """Internal stream filter that evaluates any expressions in `START` and - `TEXT` events. - """ - filters = (self._flatten, self._eval) + def _flatten(self, stream, ctxt, **vars): + number_conv = self._number_conv for kind, data, pos in stream: @@ -410,54 +545,50 @@ tag, attrs = data new_attrs = [] for name, substream in attrs: - if isinstance(substream, basestring): - value = substream - else: + if type(substream) is list: values = [] - for subkind, subdata, subpos in self._eval(substream, - ctxt): - if subkind is TEXT: - values.append(subdata) + for event in self._flatten(substream, ctxt, **vars): + if event[0] is TEXT: + values.append(event[1]) value = [x for x in values if x is not None] if not value: continue + else: + value = substream new_attrs.append((name, u''.join(value))) yield kind, (tag, Attrs(new_attrs)), pos elif kind is EXPR: - result = data.evaluate(ctxt) + result = _eval_expr(data, ctxt, **vars) if result is not None: - # First check for a string, otherwise the iterable test below - # succeeds, and the string will be chopped up into individual - # characters + # First check for a string, otherwise the iterable test + # below succeeds, and the string will be chopped up into + # individual characters if isinstance(result, basestring): yield TEXT, result, pos + elif isinstance(result, (int, float, long)): + yield TEXT, number_conv(result), pos elif hasattr(result, '__iter__'): - substream = _ensure(result) - for filter_ in filters: - substream = filter_(substream, ctxt) - for event in substream: + for event in self._flatten(_ensure(result), ctxt, + **vars): yield event else: yield TEXT, unicode(result), pos + elif kind is EXEC: + _exec_suite(data, ctxt, **vars) + + elif kind is SUB: + # This event is a list of directives and a list of nested + # events to which those directives should be applied + substream = _apply_directives(data[1], data[0], ctxt, **vars) + for event in self._flatten(substream, ctxt, **vars): + yield event + else: yield kind, data, pos - def _flatten(self, stream, ctxt): - """Internal stream filter that expands `SUB` events in the stream.""" - for event in stream: - if event[0] is SUB: - # This event is a list of directives and a list of nested - # events to which those directives should be applied - directives, substream = event[1] - substream = _apply_directives(substream, ctxt, directives) - for event in self._flatten(substream, ctxt): - yield event - else: - yield event - - def _include(self, stream, ctxt): + def _include(self, stream, ctxt, **vars): """Internal stream filter that performs inclusion of external template files. """ @@ -465,29 +596,31 @@ for event in stream: if event[0] is INCLUDE: - href, fallback = event[1] + href, cls, fallback = event[1] if not isinstance(href, basestring): parts = [] - for subkind, subdata, subpos in self._eval(href, ctxt): + for subkind, subdata, subpos in self._flatten(href, ctxt, + **vars): if subkind is TEXT: parts.append(subdata) href = u''.join([x for x in parts if x is not None]) try: tmpl = self.loader.load(href, relative_to=event[2][0], - cls=self.__class__) - for event in tmpl.generate(ctxt): + cls=cls or self.__class__) + for event in tmpl.generate(ctxt, **vars): yield event except TemplateNotFound: if fallback is None: raise for filter_ in self.filters: - fallback = filter_(iter(fallback), ctxt) + fallback = filter_(iter(fallback), ctxt, **vars) for event in fallback: yield event else: yield event +EXEC = Template.EXEC EXPR = Template.EXPR INCLUDE = Template.INCLUDE SUB = Template.SUB
--- a/genshi/template/directives.py +++ b/genshi/template/directives.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -13,13 +13,12 @@ """Implementation of the various template directives.""" -import compiler - -from genshi.core import Attrs, QName, Stream +from genshi.core import QName, Stream from genshi.path import Path from genshi.template.base import TemplateRuntimeError, TemplateSyntaxError, \ - EXPR, _apply_directives -from genshi.template.eval import Expression, _parse + EXPR, _apply_directives, _eval_expr +from genshi.template.eval import Expression, ExpressionASTTransformer, \ + _ast, _parse __all__ = ['AttrsDirective', 'ChooseDirective', 'ContentDirective', 'DefDirective', 'ForDirective', 'IfDirective', 'MatchDirective', @@ -66,7 +65,10 @@ :param template: the `Template` object :param stream: the event stream associated with the directive - :param value: the argument value for the directive + :param value: the argument value for the directive; if the directive was + specified as an element, this will be an `Attrs` instance + with all specified attributes, otherwise it will be a + `unicode` object with just the attribute value :param namespaces: a mapping of namespace URIs to prefixes :param pos: a ``(filename, lineno, offset)`` tuple describing the location where the directive was found in the source @@ -80,13 +82,15 @@ return cls(value, template, namespaces, *pos[1:]), stream attach = classmethod(attach) - def __call__(self, stream, ctxt, directives): + def __call__(self, stream, directives, ctxt, **vars): """Apply the directive to the given stream. :param stream: the event stream - :param ctxt: the context data :param directives: a list of the remaining directives that should process the stream + :param ctxt: the context data + :param vars: additional variables that should be made available when + Python code is executed """ raise NotImplementedError @@ -112,14 +116,14 @@ def _assignment(ast): - """Takes the AST representation of an assignment, and returns a function - that applies the assignment of a given value to a dictionary. + """Takes the AST representation of an assignment, and returns a + function that applies the assignment of a given value to a dictionary. """ def _names(node): - if isinstance(node, (compiler.ast.AssTuple, compiler.ast.Tuple)): - return tuple([_names(child) for child in node.nodes]) - elif isinstance(node, (compiler.ast.AssName, compiler.ast.Name)): - return node.name + if isinstance(node, _ast.Tuple): + return tuple([_names(child) for child in node.elts]) + elif isinstance(node, _ast.Name): + return node.id def _assign(data, value, names=_names(ast)): if type(names) is tuple: for idx in range(len(names)): @@ -159,10 +163,10 @@ """ __slots__ = [] - def __call__(self, stream, ctxt, directives): + def __call__(self, stream, directives, ctxt, **vars): def _generate(): kind, (tag, attrib), pos = stream.next() - attrs = self.expr.evaluate(ctxt) + attrs = _eval_expr(self.expr, ctxt, **vars) if attrs: if isinstance(attrs, Stream): try: @@ -178,7 +182,7 @@ for event in stream: yield event - return _apply_directives(_generate(), ctxt, directives) + return _apply_directives(_generate(), directives, ctxt, **vars) class ContentDirective(Directive): @@ -199,6 +203,10 @@ __slots__ = [] def attach(cls, template, stream, value, namespaces, pos): + if type(value) is dict: + raise TemplateSyntaxError('The content directive can not be used ' + 'as an element', template.filepath, + *pos[1:]) expr = cls._parse_expr(value, template, *pos[1:]) return None, [stream[0], (EXPR, expr, pos), stream[-1]] attach = classmethod(attach) @@ -248,35 +256,39 @@ __slots__ = ['name', 'args', 'star_args', 'dstar_args', 'defaults', 'signature'] - ATTRIBUTE = 'function' - def __init__(self, args, template, namespaces=None, lineno=-1, offset=-1): Directive.__init__(self, None, template, namespaces, lineno, offset) self.signature = args.strip() - ast = _parse(args).node + ast = _parse(args).body self.args = [] self.star_args = None self.dstar_args = None self.defaults = {} - if isinstance(ast, compiler.ast.CallFunc): - self.name = ast.node.name + if isinstance(ast, _ast.Call): + self.name = ast.func.id for arg in ast.args: - if isinstance(arg, compiler.ast.Keyword): - self.args.append(arg.name) - self.defaults[arg.name] = Expression(arg.expr, - template.filepath, - lineno, - lookup=template.lookup) - else: - self.args.append(arg.name) - if ast.star_args: - self.star_args = ast.star_args.name - if ast.dstar_args: - self.dstar_args = ast.dstar_args.name + # only names + self.args.append(arg.id) + for kwd in ast.keywords: + self.args.append(kwd.arg) + exp = Expression(kwd.value, template.filepath, + lineno, lookup=template.lookup) + self.defaults[kwd.arg] = exp + if getattr(ast, 'starargs', None): + self.star_args = ast.starargs.id + if getattr(ast, 'kwargs', None): + self.dstar_args = ast.kwargs.id else: - self.name = ast.name + self.name = ast.id - def __call__(self, stream, ctxt, directives): + def attach(cls, template, stream, value, namespaces, pos): + if type(value) is dict: + value = value.get('function') + return super(DefDirective, cls).attach(template, stream, value, + namespaces, pos) + attach = classmethod(attach) + + def __call__(self, stream, directives, ctxt, **vars): stream = list(stream) def function(*args, **kwargs): @@ -289,21 +301,17 @@ if name in kwargs: val = kwargs.pop(name) else: - val = self.defaults.get(name).evaluate(ctxt) + val = _eval_expr(self.defaults.get(name), ctxt, **vars) scope[name] = val if not self.star_args is None: scope[self.star_args] = args if not self.dstar_args is None: scope[self.dstar_args] = kwargs ctxt.push(scope) - for event in _apply_directives(stream, ctxt, directives): + for event in _apply_directives(stream, directives, ctxt, **vars): yield event ctxt.pop() - try: - function.__name__ = self.name - except TypeError: - # Function name can't be set in Python 2.3 - pass + function.__name__ = self.name # Store the function reference in the bottom context frame so that it # doesn't get popped off before processing the template has finished @@ -331,37 +339,38 @@ """ __slots__ = ['assign', 'target', 'filename'] - ATTRIBUTE = 'each' - def __init__(self, value, template, namespaces=None, lineno=-1, offset=-1): if ' in ' not in value: raise TemplateSyntaxError('"in" keyword missing in "for" directive', template.filepath, lineno, offset) assign, value = value.split(' in ', 1) - self.target = _parse(assign, 'exec').node.nodes[0].expr - self.assign = _assignment(self.target) + self.target = _parse(assign, 'exec') + self.assign = _assignment(self.target.body[0].value) + value = 'iter(%s)' % value.strip() self.filename = template.filepath - Directive.__init__(self, value.strip(), template, namespaces, lineno, - offset) + Directive.__init__(self, value, template, namespaces, lineno, offset) - def __call__(self, stream, ctxt, directives): - iterable = self.expr.evaluate(ctxt) + def attach(cls, template, stream, value, namespaces, pos): + if type(value) is dict: + value = value.get('each') + return super(ForDirective, cls).attach(template, stream, value, + namespaces, pos) + attach = classmethod(attach) + + def __call__(self, stream, directives, ctxt, **vars): + iterable = _eval_expr(self.expr, ctxt, **vars) if iterable is None: return assign = self.assign scope = {} stream = list(stream) - try: - iterator = iter(iterable) - for item in iterator: - assign(scope, item) - ctxt.push(scope) - for event in _apply_directives(stream, ctxt, directives): - yield event - ctxt.pop() - except TypeError, e: - raise TemplateRuntimeError(str(e), self.filename, *stream[0][2][1:]) + for item in iterable: + assign(scope, item) + ctxt.push(scope) + for event in _apply_directives(stream, directives, ctxt, **vars): + yield event + ctxt.pop() def __repr__(self): return '<%s>' % self.__class__.__name__ @@ -382,11 +391,17 @@ """ __slots__ = [] - ATTRIBUTE = 'test' + def attach(cls, template, stream, value, namespaces, pos): + if type(value) is dict: + value = value.get('test') + return super(IfDirective, cls).attach(template, stream, value, + namespaces, pos) + attach = classmethod(attach) - def __call__(self, stream, ctxt, directives): - if self.expr.evaluate(ctxt): - return _apply_directives(stream, ctxt, directives) + def __call__(self, stream, directives, ctxt, **vars): + value = _eval_expr(self.expr, ctxt, **vars) + if value: + return _apply_directives(stream, directives, ctxt, **vars) return [] @@ -407,19 +422,33 @@ </span> </div> """ - __slots__ = ['path', 'namespaces'] + __slots__ = ['path', 'namespaces', 'hints'] - ATTRIBUTE = 'path' - - def __init__(self, value, template, namespaces=None, lineno=-1, offset=-1): + def __init__(self, value, template, hints=None, namespaces=None, + lineno=-1, offset=-1): Directive.__init__(self, None, template, namespaces, lineno, offset) self.path = Path(value, template.filepath, lineno) self.namespaces = namespaces or {} + self.hints = hints or () - def __call__(self, stream, ctxt, directives): + def attach(cls, template, stream, value, namespaces, pos): + hints = [] + if type(value) is dict: + if value.get('buffer', '').lower() == 'false': + hints.append('not_buffered') + if value.get('once', '').lower() == 'true': + hints.append('match_once') + if value.get('recursive', '').lower() == 'false': + hints.append('not_recursive') + value = value.get('path') + return cls(value, template, frozenset(hints), namespaces, *pos[1:]), \ + stream + attach = classmethod(attach) + + def __call__(self, stream, directives, ctxt, **vars): ctxt._match_templates.append((self.path.test(ignore_context=True), - self.path, list(stream), self.namespaces, - directives)) + self.path, list(stream), self.hints, + self.namespaces, directives)) return [] def __repr__(self): @@ -455,6 +484,8 @@ __slots__ = [] def attach(cls, template, stream, value, namespaces, pos): + if type(value) is dict: + value = value.get('value') if not value: raise TemplateSyntaxError('missing value for "replace" directive', template.filepath, *pos[1:]) @@ -496,9 +527,9 @@ """ __slots__ = [] - def __call__(self, stream, ctxt, directives): + def __call__(self, stream, directives, ctxt, **vars): def _generate(): - if self.expr.evaluate(ctxt): + if _eval_expr(self.expr, ctxt, **vars): stream.next() # skip start tag previous = stream.next() for event in stream: @@ -507,7 +538,7 @@ else: for event in stream: yield event - return _apply_directives(_generate(), ctxt, directives) + return _apply_directives(_generate(), directives, ctxt, **vars) def attach(cls, template, stream, value, namespaces, pos): if not value: @@ -558,16 +589,21 @@ """ __slots__ = ['matched', 'value'] - ATTRIBUTE = 'test' + def attach(cls, template, stream, value, namespaces, pos): + if type(value) is dict: + value = value.get('test') + return super(ChooseDirective, cls).attach(template, stream, value, + namespaces, pos) + attach = classmethod(attach) - def __call__(self, stream, ctxt, directives): - frame = dict({'_choose.matched': False}) + def __call__(self, stream, directives, ctxt, **vars): + info = [False, bool(self.expr), None] if self.expr: - frame['_choose.value'] = self.expr.evaluate(ctxt) - ctxt.push(frame) - for event in _apply_directives(stream, ctxt, directives): + info[2] = _eval_expr(self.expr, ctxt, **vars) + ctxt._choice_stack.append(info) + for event in _apply_directives(stream, directives, ctxt, **vars): yield event - ctxt.pop() + ctxt._choice_stack.pop() class WhenDirective(Directive): @@ -578,37 +614,42 @@ """ __slots__ = ['filename'] - ATTRIBUTE = 'test' - def __init__(self, value, template, namespaces=None, lineno=-1, offset=-1): Directive.__init__(self, value, template, namespaces, lineno, offset) self.filename = template.filepath - def __call__(self, stream, ctxt, directives): - matched, frame = ctxt._find('_choose.matched') - if not frame: + def attach(cls, template, stream, value, namespaces, pos): + if type(value) is dict: + value = value.get('test') + return super(WhenDirective, cls).attach(template, stream, value, + namespaces, pos) + attach = classmethod(attach) + + def __call__(self, stream, directives, ctxt, **vars): + info = ctxt._choice_stack and ctxt._choice_stack[-1] + if not info: raise TemplateRuntimeError('"when" directives can only be used ' 'inside a "choose" directive', self.filename, *stream.next()[2][1:]) - if matched: + if info[0]: return [] - if not self.expr and '_choose.value' not in frame: + if not self.expr and not info[1]: raise TemplateRuntimeError('either "choose" or "when" directive ' 'must have a test expression', self.filename, *stream.next()[2][1:]) - if '_choose.value' in frame: - value = frame['_choose.value'] + if info[1]: + value = info[2] if self.expr: - matched = value == self.expr.evaluate(ctxt) + matched = value == _eval_expr(self.expr, ctxt, **vars) else: matched = bool(value) else: - matched = bool(self.expr.evaluate(ctxt)) - frame['_choose.matched'] = matched + matched = bool(_eval_expr(self.expr, ctxt, **vars)) + info[0] = matched if not matched: return [] - return _apply_directives(stream, ctxt, directives) + return _apply_directives(stream, directives, ctxt, **vars) class OtherwiseDirective(Directive): @@ -623,17 +664,17 @@ Directive.__init__(self, None, template, namespaces, lineno, offset) self.filename = template.filepath - def __call__(self, stream, ctxt, directives): - matched, frame = ctxt._find('_choose.matched') - if not frame: + def __call__(self, stream, directives, ctxt, **vars): + info = ctxt._choice_stack and ctxt._choice_stack[-1] + if not info: raise TemplateRuntimeError('an "otherwise" directive can only be ' 'used inside a "choose" directive', self.filename, *stream.next()[2][1:]) - if matched: + if info[0]: return [] - frame['_choose.matched'] = True + info[0] = True - return _apply_directives(stream, ctxt, directives) + return _apply_directives(stream, directives, ctxt, **vars) class WithDirective(Directive): @@ -651,23 +692,19 @@ """ __slots__ = ['vars'] - ATTRIBUTE = 'vars' - def __init__(self, value, template, namespaces=None, lineno=-1, offset=-1): Directive.__init__(self, None, template, namespaces, lineno, offset) self.vars = [] value = value.strip() try: - ast = _parse(value, 'exec').node - for node in ast.nodes: - if isinstance(node, compiler.ast.Discard): - continue - elif not isinstance(node, compiler.ast.Assign): + ast = _parse(value, 'exec') + for node in ast.body: + if not isinstance(node, _ast.Assign): raise TemplateSyntaxError('only assignment allowed in ' 'value of the "with" directive', template.filepath, lineno, offset) - self.vars.append(([(n, _assignment(n)) for n in node.nodes], - Expression(node.expr, template.filepath, + self.vars.append(([(n, _assignment(n)) for n in node.targets], + Expression(node.value, template.filepath, lineno, lookup=template.lookup))) except SyntaxError, err: err.msg += ' in expression "%s" of "%s" directive' % (value, @@ -675,14 +712,21 @@ raise TemplateSyntaxError(err, template.filepath, lineno, offset + (err.offset or 0)) - def __call__(self, stream, ctxt, directives): + def attach(cls, template, stream, value, namespaces, pos): + if type(value) is dict: + value = value.get('vars') + return super(WithDirective, cls).attach(template, stream, value, + namespaces, pos) + attach = classmethod(attach) + + def __call__(self, stream, directives, ctxt, **vars): frame = {} ctxt.push(frame) for targets, expr in self.vars: - value = expr.evaluate(ctxt) + value = _eval_expr(expr, ctxt, **vars) for _, assign in targets: assign(frame, value) - for event in _apply_directives(stream, ctxt, directives): + for event in _apply_directives(stream, directives, ctxt, **vars): yield event ctxt.pop()
--- a/genshi/template/eval.py +++ b/genshi/template/eval.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -14,16 +14,13 @@ """Support for "safe" evaluation of Python expressions.""" import __builtin__ -from compiler import ast, parse -from compiler.pycodegen import ExpressionCodeGenerator, ModuleCodeGenerator -import new -try: - set -except NameError: - from sets import Set as set -import sys + +from textwrap import dedent +from types import CodeType from genshi.core import Markup +from genshi.template.astutil import ASTTransformer, ASTCodeGenerator, \ + _ast, parse from genshi.template.base import TemplateRuntimeError from genshi.util import flatten @@ -32,11 +29,35 @@ __docformat__ = 'restructuredtext en' +# Check for a Python 2.4 bug in the eval loop +has_star_import_bug = False +try: + class _FakeMapping(object): + __getitem__ = __setitem__ = lambda *a: None + exec 'from sys import *' in {}, _FakeMapping() +except SystemError: + has_star_import_bug = True +del _FakeMapping + + +def _star_import_patch(mapping, modname): + """This function is used as helper if a Python version with a broken + star-import opcode is in use. + """ + module = __import__(modname, None, None, ['__all__']) + if hasattr(module, '__all__'): + members = module.__all__ + else: + members = [x for x in module.__dict__ if not x.startswith('_')] + mapping.update([(name, getattr(module, name)) for name in members]) + + class Code(object): """Abstract base class for the `Expression` and `Suite` classes.""" - __slots__ = ['source', 'code', '_globals'] + __slots__ = ['source', 'code', 'ast', '_globals'] - def __init__(self, source, filename=None, lineno=-1, lookup='lenient'): + def __init__(self, source, filename=None, lineno=-1, lookup='strict', + xform=None): """Create the code object, either from a string, or from an AST node. :param source: either a string containing the source code, or an AST @@ -45,27 +66,49 @@ the code :param lineno: the number of the line on which the code was found :param lookup: the lookup class that defines how variables are looked - up in the context. Can be either `LenientLookup` (the - default), `StrictLookup`, or a custom lookup class + up in the context; can be either "strict" (the default), + "lenient", or a custom lookup class + :param xform: the AST transformer that should be applied to the code; + if `None`, the appropriate transformation is chosen + depending on the mode """ if isinstance(source, basestring): self.source = source node = _parse(source, mode=self.mode) else: - assert isinstance(source, ast.Node) + assert isinstance(source, _ast.AST), \ + 'Expected string or AST node, but got %r' % source self.source = '?' if self.mode == 'eval': - node = ast.Expression(source) + node = _ast.Expression() + node.body = source else: - node = ast.Module(None, source) + node = _ast.Module() + node.body = [source] + self.ast = node self.code = _compile(node, self.source, mode=self.mode, - filename=filename, lineno=lineno) + filename=filename, lineno=lineno, xform=xform) if lookup is None: lookup = LenientLookup elif isinstance(lookup, basestring): lookup = {'lenient': LenientLookup, 'strict': StrictLookup}[lookup] - self._globals = lookup.globals() + self._globals = lookup.globals + + def __getstate__(self): + state = {'source': self.source, 'ast': self.ast, + 'lookup': self._globals.im_self} + c = self.code + state['code'] = (c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code, + c.co_consts, c.co_names, c.co_varnames, c.co_filename, + c.co_name, c.co_firstlineno, c.co_lnotab, (), ()) + return state + + def __setstate__(self, state): + self.source = state['source'] + self.ast = state['ast'] + self.code = CodeType(0, *state['code']) + self._globals = state['lookup'].globals def __eq__(self, other): return (type(other) == type(self)) and (self.code == other.code) @@ -131,9 +174,8 @@ :return: the result of the evaluation """ __traceback_hide__ = 'before_and_this' - _globals = self._globals - _globals['data'] = data - return eval(self.code, _globals, {'data': data}) + _globals = self._globals(data) + return eval(self.code, _globals, {'__data__': data}) class Suite(Code): @@ -153,8 +195,7 @@ :param data: a mapping containing the data to execute in """ __traceback_hide__ = 'before_and_this' - _globals = self._globals - _globals['data'] = data + _globals = self._globals(data) exec self.code in _globals, data @@ -240,14 +281,17 @@ class LookupBase(object): """Abstract base class for variable lookup implementations.""" - def globals(cls): + def globals(cls, data): """Construct the globals dictionary to use as the execution context for the expression or suite. """ return { + '__data__': data, '_lookup_name': cls.lookup_name, '_lookup_attr': cls.lookup_attr, - '_lookup_item': cls.lookup_item + '_lookup_item': cls.lookup_item, + '_star_import_patch': _star_import_patch, + 'UndefinedError': UndefinedError, } globals = classmethod(globals) @@ -257,21 +301,26 @@ if val is UNDEFINED: val = BUILTINS.get(name, val) if val is UNDEFINED: - return cls.undefined(name) + val = cls.undefined(name) return val lookup_name = classmethod(lookup_name) - def lookup_attr(cls, data, obj, key): + def lookup_attr(cls, obj, key): __traceback_hide__ = True - if hasattr(obj, key): - return getattr(obj, key) try: - return obj[key] - except (KeyError, TypeError): - return cls.undefined(key, owner=obj) + val = getattr(obj, key) + except AttributeError: + if hasattr(obj.__class__, key): + raise + else: + try: + val = obj[key] + except (KeyError, TypeError): + val = cls.undefined(key, owner=obj) + return val lookup_attr = classmethod(lookup_attr) - def lookup_item(cls, data, obj, key): + def lookup_item(cls, obj, key): __traceback_hide__ = True if len(key) == 1: key = key[0] @@ -281,7 +330,7 @@ if isinstance(key, basestring): val = getattr(obj, key, UNDEFINED) if val is UNDEFINED: - return cls.undefined(key, owner=obj) + val = cls.undefined(key, owner=obj) return val raise lookup_item = classmethod(lookup_item) @@ -356,296 +405,75 @@ def _parse(source, mode='eval'): + source = source.strip() + if mode == 'exec': + lines = [line.expandtabs() for line in source.splitlines()] + if lines: + first = lines[0] + rest = dedent('\n'.join(lines[1:])).rstrip() + if first.rstrip().endswith(':') and not rest[0].isspace(): + rest = '\n'.join([' %s' % line for line in rest.splitlines()]) + source = '\n'.join([first, rest]) if isinstance(source, unicode): source = '\xef\xbb\xbf' + source.encode('utf-8') return parse(source, mode) -def _compile(node, source=None, mode='eval', filename=None, lineno=-1): - xform = {'eval': ExpressionASTTransformer}.get(mode, TemplateASTTransformer) - tree = xform().visit(node) + +def _compile(node, source=None, mode='eval', filename=None, lineno=-1, + xform=None): if isinstance(filename, unicode): # unicode file names not allowed for code objects filename = filename.encode('utf-8', 'replace') elif not filename: filename = '<string>' - tree.filename = filename if lineno <= 0: lineno = 1 + if xform is None: + xform = { + 'eval': ExpressionASTTransformer + }.get(mode, TemplateASTTransformer) + tree = xform().visit(node) + if mode == 'eval': - gen = ExpressionCodeGenerator(tree) - name = '<Expression %s>' % (repr(source or '?')) + name = '<Expression %r>' % (source or '?') else: - gen = ModuleCodeGenerator(tree) - name = '<Suite>' - gen.optimized = True - code = gen.getCode() + lines = source.splitlines() + if not lines: + extract = '' + else: + extract = lines[0] + if len(lines) > 1: + extract += ' ...' + name = '<Suite %r>' % (extract) + new_source = ASTCodeGenerator(tree).code + code = compile(new_source, filename, mode) - # We'd like to just set co_firstlineno, but it's readonly. So we need to - # clone the code object while adjusting the line number - return new.code(0, code.co_nlocals, code.co_stacksize, - code.co_flags | 0x0040, code.co_code, code.co_consts, - code.co_names, code.co_varnames, filename, name, lineno, - code.co_lnotab, (), ()) + try: + # We'd like to just set co_firstlineno, but it's readonly. So we need + # to clone the code object while adjusting the line number + return CodeType(0, code.co_nlocals, code.co_stacksize, + code.co_flags | 0x0040, code.co_code, code.co_consts, + code.co_names, code.co_varnames, filename, name, + lineno, code.co_lnotab, (), ()) + except RuntimeError: + return code + + +def _new(class_, *args, **kwargs): + ret = class_() + for attr, value in zip(ret._fields, args): + if attr in kwargs: + raise ValueError('Field set both in args and kwargs') + setattr(ret, attr, value) + for attr, value in kwargs: + setattr(ret, attr, value) + return ret + BUILTINS = __builtin__.__dict__.copy() BUILTINS.update({'Markup': Markup, 'Undefined': Undefined}) - - -class ASTTransformer(object): - """General purpose base class for AST transformations. - - Every visitor method can be overridden to return an AST node that has been - altered or replaced in some way. - """ - - def visit(self, node): - if node is None: - return None - if type(node) is tuple: - return tuple([self.visit(n) for n in node]) - visitor = getattr(self, 'visit%s' % node.__class__.__name__, - self._visitDefault) - return visitor(node) - - def _visitDefault(self, node): - return node - - def visitExpression(self, node): - node.node = self.visit(node.node) - return node - - def visitModule(self, node): - node.node = self.visit(node.node) - return node - - def visitStmt(self, node): - node.nodes = [self.visit(x) for x in node.nodes] - return node - - # Classes, Functions & Accessors - - def visitCallFunc(self, node): - node.node = self.visit(node.node) - node.args = [self.visit(x) for x in node.args] - if node.star_args: - node.star_args = self.visit(node.star_args) - if node.dstar_args: - node.dstar_args = self.visit(node.dstar_args) - return node - - def visitClass(self, node): - node.bases = [self.visit(x) for x in node.bases] - node.code = self.visit(node.code) - node.filename = '<string>' # workaround for bug in pycodegen - return node - - def visitFunction(self, node): - if hasattr(node, 'decorators'): - node.decorators = self.visit(node.decorators) - node.defaults = [self.visit(x) for x in node.defaults] - node.code = self.visit(node.code) - node.filename = '<string>' # workaround for bug in pycodegen - return node - - def visitGetattr(self, node): - node.expr = self.visit(node.expr) - return node - - def visitLambda(self, node): - node.code = self.visit(node.code) - node.filename = '<string>' # workaround for bug in pycodegen - return node - - def visitSubscript(self, node): - node.expr = self.visit(node.expr) - node.subs = [self.visit(x) for x in node.subs] - return node - - # Statements - - def visitAssert(self, node): - node.test = self.visit(node.test) - node.fail = self.visit(node.fail) - return node - - def visitAssign(self, node): - node.nodes = [self.visit(x) for x in node.nodes] - node.expr = self.visit(node.expr) - return node - - def visitAssAttr(self, node): - node.expr = self.visit(node.expr) - return node - - def visitAugAssign(self, node): - node.node = self.visit(node.node) - node.expr = self.visit(node.expr) - return node - - def visitDecorators(self, node): - node.nodes = [self.visit(x) for x in node.nodes] - return node - - def visitExec(self, node): - node.expr = self.visit(node.expr) - node.locals = self.visit(node.locals) - node.globals = self.visit(node.globals) - return node - - def visitFor(self, node): - node.assign = self.visit(node.assign) - node.list = self.visit(node.list) - node.body = self.visit(node.body) - node.else_ = self.visit(node.else_) - return node - - def visitIf(self, node): - node.tests = [self.visit(x) for x in node.tests] - node.else_ = self.visit(node.else_) - return node - - def _visitPrint(self, node): - node.nodes = [self.visit(x) for x in node.nodes] - node.dest = self.visit(node.dest) - return node - visitPrint = visitPrintnl = _visitPrint - - def visitRaise(self, node): - node.expr1 = self.visit(node.expr1) - node.expr2 = self.visit(node.expr2) - node.expr3 = self.visit(node.expr3) - return node - - def visitReturn(self, node): - node.value = self.visit(node.value) - return node - - def visitTryExcept(self, node): - node.body = self.visit(node.body) - node.handlers = self.visit(node.handlers) - node.else_ = self.visit(node.else_) - return node - - def visitTryFinally(self, node): - node.body = self.visit(node.body) - node.final = self.visit(node.final) - return node - - def visitWhile(self, node): - node.test = self.visit(node.test) - node.body = self.visit(node.body) - node.else_ = self.visit(node.else_) - return node - - def visitWith(self, node): - node.expr = self.visit(node.expr) - node.vars = [self.visit(x) for x in node.vars] - node.body = self.visit(node.body) - return node - - def visitYield(self, node): - node.value = self.visit(node.value) - return node - - # Operators - - def _visitBoolOp(self, node): - node.nodes = [self.visit(x) for x in node.nodes] - return node - visitAnd = visitOr = visitBitand = visitBitor = visitBitxor = _visitBoolOp - visitAssTuple = visitAssList = _visitBoolOp - - def _visitBinOp(self, node): - node.left = self.visit(node.left) - node.right = self.visit(node.right) - return node - visitAdd = visitSub = _visitBinOp - visitDiv = visitFloorDiv = visitMod = visitMul = visitPower = _visitBinOp - visitLeftShift = visitRightShift = _visitBinOp - - def visitCompare(self, node): - node.expr = self.visit(node.expr) - node.ops = [(op, self.visit(n)) for op, n in node.ops] - return node - - def _visitUnaryOp(self, node): - node.expr = self.visit(node.expr) - return node - visitUnaryAdd = visitUnarySub = visitNot = visitInvert = _visitUnaryOp - visitBackquote = visitDiscard = _visitUnaryOp - - def visitIfExp(self, node): - node.test = self.visit(node.test) - node.then = self.visit(node.then) - node.else_ = self.visit(node.else_) - return node - - # Identifiers, Literals and Comprehensions - - def visitDict(self, node): - node.items = [(self.visit(k), - self.visit(v)) for k, v in node.items] - return node - - def visitGenExpr(self, node): - node.code = self.visit(node.code) - node.filename = '<string>' # workaround for bug in pycodegen - return node - - def visitGenExprFor(self, node): - node.assign = self.visit(node.assign) - node.iter = self.visit(node.iter) - node.ifs = [self.visit(x) for x in node.ifs] - return node - - def visitGenExprIf(self, node): - node.test = self.visit(node.test) - return node - - def visitGenExprInner(self, node): - node.quals = [self.visit(x) for x in node.quals] - node.expr = self.visit(node.expr) - return node - - def visitKeyword(self, node): - node.expr = self.visit(node.expr) - return node - - def visitList(self, node): - node.nodes = [self.visit(n) for n in node.nodes] - return node - - def visitListComp(self, node): - node.quals = [self.visit(x) for x in node.quals] - node.expr = self.visit(node.expr) - return node - - def visitListCompFor(self, node): - node.assign = self.visit(node.assign) - node.list = self.visit(node.list) - node.ifs = [self.visit(x) for x in node.ifs] - return node - - def visitListCompIf(self, node): - node.test = self.visit(node.test) - return node - - def visitSlice(self, node): - node.expr = self.visit(node.expr) - if node.lower is not None: - node.lower = self.visit(node.lower) - if node.upper is not None: - node.upper = self.visit(node.upper) - return node - - def visitSliceobj(self, node): - node.nodes = [self.visit(x) for x in node.nodes] - return node - - def visitTuple(self, node): - node.nodes = [self.visit(n) for n in node.nodes] - return node +CONSTANTS = frozenset(['False', 'True', 'None', 'NotImplemented', 'Ellipsis']) class TemplateASTTransformer(ASTTransformer): @@ -654,81 +482,121 @@ """ def __init__(self): - self.locals = [] - - def visitConst(self, node): - if isinstance(node.value, str): - try: # If the string is ASCII, return a `str` object - node.value.decode('ascii') - except ValueError: # Otherwise return a `unicode` object - return ast.Const(node.value.decode('utf-8')) - return node - - def visitAssName(self, node): - if self.locals: - self.locals[-1].add(node.name) - return node + self.locals = [CONSTANTS] - def visitAugAssign(self, node): - if isinstance(node.node, ast.Name): - name = node.node.name - node.node = ast.Subscript(ast.Name('data'), 'OP_APPLY', - [ast.Const(name)]) - node.expr = self.visit(node.expr) - return ast.If([ - (ast.Compare(ast.Const(name), [('in', ast.Name('data'))]), - ast.Stmt([node]))], - ast.Stmt([ast.Raise(ast.CallFunc(ast.Name('UndefinedError'), - [ast.Const(name)]), - None, None)])) - else: - return ASTTransformer.visitAugAssign(self, node) + def _extract_names(self, node): + names = set() + def _process(node): + if isinstance(node, _ast.Name): + names.add(node.id) + elif isinstance(node, _ast.alias): + names.add(node.asname or node.name) + elif isinstance(node, _ast.Tuple): + for elt in node.elts: + _process(node) + if hasattr(node, 'args'): + for arg in node.args: + _process(arg) + if hasattr(node, 'vararg'): + names.add(node.vararg) + if hasattr(node, 'kwarg'): + names.add(node.kwarg) + elif hasattr(node, 'names'): + for elt in node.names: + _process(elt) + return names - def visitClass(self, node): - self.locals.append(set()) - node = ASTTransformer.visitClass(self, node) - self.locals.pop() + def visit_Str(self, node): + if isinstance(node.s, str): + try: # If the string is ASCII, return a `str` object + node.s.decode('ascii') + except ValueError: # Otherwise return a `unicode` object + return _new(_ast.Str, node.s.decode('utf-8')) return node - def visitFor(self, node): - self.locals.append(set()) - node = ASTTransformer.visitFor(self, node) - self.locals.pop() - return node - - def visitFunction(self, node): - self.locals.append(set(node.argnames)) - node = ASTTransformer.visitFunction(self, node) - self.locals.pop() - return node - - def visitGenExpr(self, node): + def visit_ClassDef(self, node): + if len(self.locals) > 1: + self.locals[-1].add(node.name) self.locals.append(set()) - node = ASTTransformer.visitGenExpr(self, node) - self.locals.pop() - return node + try: + return ASTTransformer.visit_ClassDef(self, node) + finally: + self.locals.pop() - def visitLambda(self, node): - self.locals.append(set(flatten(node.argnames))) - node = ASTTransformer.visitLambda(self, node) - self.locals.pop() - return node + def visit_Import(self, node): + if len(self.locals) > 1: + self.locals[-1].update(self._extract_names(node)) + return ASTTransformer.visit_Import(self, node) - def visitListComp(self, node): - self.locals.append(set()) - node = ASTTransformer.visitListComp(self, node) - self.locals.pop() - return node + def visit_ImportFrom(self, node): + if [a.name for a in node.names] == ['*']: + if has_star_import_bug: + # This is a Python 2.4 bug. Only if we have a broken Python + # version do we need to apply this hack + node = _new(_ast.Expr, _new(_ast.Call, + _new(_ast.Name, '_star_import_patch'), [ + _new(_ast.Name, '__data__'), + _new(_ast.Str, node.module) + ], (), ())) + return node + if len(self.locals) > 1: + self.locals[-1].update(self._extract_names(node)) + return ASTTransformer.visit_ImportFrom(self, node) - def visitName(self, node): + def visit_FunctionDef(self, node): + if len(self.locals) > 1: + self.locals[-1].add(node.name) + + self.locals.append(self._extract_names(node.args)) + try: + return ASTTransformer.visit_FunctionDef(self, node) + finally: + self.locals.pop() + + # GeneratorExp(expr elt, comprehension* generators) + def visit_GeneratorExp(self, node): + gens = [] + # need to visit them in inverse order + for generator in node.generators[::-1]: + # comprehension = (expr target, expr iter, expr* ifs) + self.locals.append(set()) + gen = _new(_ast.comprehension, self.visit(generator.target), + self.visit(generator.iter), + [self.visit(if_) for if_ in generator.ifs]) + gens.append(gen) + gens.reverse() + + # use node.__class__ to make it reusable as ListComp + ret = _new(node.__class__, self.visit(node.elt), gens) + #delete inserted locals + del self.locals[-len(node.generators):] + return ret + + # ListComp(expr elt, comprehension* generators) + visit_ListComp = visit_GeneratorExp + + def visit_Lambda(self, node): + self.locals.append(self._extract_names(node.args)) + try: + return ASTTransformer.visit_Lambda(self, node) + finally: + self.locals.pop() + + def visit_Name(self, node): # If the name refers to a local inside a lambda, list comprehension, or # generator expression, leave it alone - for frame in self.locals: - if node.name in frame: - return node - # Otherwise, translate the name ref into a context lookup - func_args = [ast.Name('data'), ast.Const(node.name)] - return ast.CallFunc(ast.Name('_lookup_name'), func_args) + if isinstance(node.ctx, _ast.Load) and \ + node.id not in flatten(self.locals): + # Otherwise, translate the name ref into a context lookup + name = _new(_ast.Name, '_lookup_name', _ast.Load()) + namearg = _new(_ast.Name, '__data__', _ast.Load()) + strarg = _new(_ast.Str, node.id) + node = _new(_ast.Call, name, [namearg, strarg], []) + elif isinstance(node.ctx, _ast.Store): + if len(self.locals) > 1: + self.locals[-1].add(node.id) + + return node class ExpressionASTTransformer(TemplateASTTransformer): @@ -736,14 +604,22 @@ for code embedded in templates. """ - def visitGetattr(self, node): - return ast.CallFunc(ast.Name('_lookup_attr'), [ - ast.Name('data'), self.visit(node.expr), - ast.Const(node.attrname) - ]) + def visit_Attribute(self, node): + if not isinstance(node.ctx, _ast.Load): + return ASTTransformer.visit_Attribute(self, node) - def visitSubscript(self, node): - return ast.CallFunc(ast.Name('_lookup_item'), [ - ast.Name('data'), self.visit(node.expr), - ast.Tuple([self.visit(sub) for sub in node.subs]) - ]) + func = _new(_ast.Name, '_lookup_attr', _ast.Load()) + args = [self.visit(node.value), _new(_ast.Str, node.attr)] + return _new(_ast.Call, func, args, []) + + def visit_Subscript(self, node): + if not isinstance(node.ctx, _ast.Load) or \ + not isinstance(node.slice, _ast.Index): + return ASTTransformer.visit_Subscript(self, node) + + func = _new(_ast.Name, '_lookup_item', _ast.Load()) + args = [ + self.visit(node.value), + _new(_ast.Tuple, (self.visit(node.slice.value),), _ast.Load()) + ] + return _new(_ast.Call, func, args, [])
--- a/genshi/template/inline.py +++ b/genshi/template/inline.py @@ -15,7 +15,7 @@ import imp from genshi.core import Attrs, Stream, _ensure, START, END, TEXT -from genshi.template.core import EXPR, SUB +from genshi.template.base import EXPR, SUB from genshi.template.directives import *
--- a/genshi/template/interpolation.py +++ b/genshi/template/interpolation.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2007 Edgewall Software +# Copyright (C) 2007-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -17,7 +17,8 @@ from itertools import chain import os -from tokenize import tokenprog +import re +from tokenize import PseudoToken from genshi.core import TEXT from genshi.template.base import TemplateSyntaxError, EXPR @@ -30,8 +31,12 @@ NAMECHARS = NAMESTART + '.0123456789' PREFIX = '$' -def interpolate(text, basedir=None, filename=None, lineno=-1, offset=0, - lookup='lenient'): +token_re = re.compile('%s|%s(?s)' % ( + r'[uU]?[rR]?("""|\'\'\')((?<!\\)\\\1|.)*?\1', + PseudoToken +)) + +def interpolate(text, filepath=None, lineno=-1, offset=0, lookup='strict'): """Parse the given string and extract expressions. This function is a generator that yields `TEXT` events for literal strings, @@ -39,15 +44,14 @@ string. >>> for kind, data, pos in interpolate("hey ${foo}bar"): - ... print kind, `data` + ... print kind, repr(data) TEXT u'hey ' EXPR Expression('foo') TEXT u'bar' :param text: the text to parse - :param basedir: base directory of the file in which the text was found - (optional) - :param filename: basename of the file in which the text was found (optional) + :param filepath: absolute path to the file in which the text was found + (optional) :param lineno: the line number at which the text was found (optional) :param offset: the column number at which the text starts in the source (optional) @@ -57,9 +61,6 @@ :raise TemplateSyntaxError: when a syntax error in an expression is encountered """ - filepath = filename - if filepath and basedir: - filepath = os.path.join(basedir, filepath) pos = [filepath, lineno, offset] textbuf = [] @@ -73,7 +74,7 @@ if chunk: try: expr = Expression(chunk.strip(), pos[0], pos[1], - lookup=lookup) + lookup=lookup) yield EXPR, expr, tuple(pos) except SyntaxError, err: raise TemplateSyntaxError(err, filepath, pos[1], @@ -111,7 +112,7 @@ pos = offset + 2 level = 1 while level: - match = tokenprog.match(text, pos) + match = token_re.match(text, pos) if match is None: raise TemplateSyntaxError('invalid syntax', filepath, *textpos[1:]) @@ -137,6 +138,8 @@ yield True, text[offset + 1:pos].strip() elif not escaped and next == PREFIX: + if offset > pos: + yield False, text[pos:offset] escaped = True pos = offset + 1
--- a/genshi/template/loader.py +++ b/genshi/template/loader.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -22,7 +22,8 @@ from genshi.template.base import TemplateError from genshi.util import LRUCache -__all__ = ['TemplateLoader', 'TemplateNotFound'] +__all__ = ['TemplateLoader', 'TemplateNotFound', 'directory', 'package', + 'prefixed'] __docformat__ = 'restructuredtext en' @@ -69,16 +70,23 @@ >>> loader.load(os.path.basename(path)) is template True + The `auto_reload` option can be used to control whether a template should + be automatically reloaded when the file it was loaded from has been + changed. Disable this automatic reloading to improve performance. + >>> os.remove(path) """ def __init__(self, search_path=None, auto_reload=False, default_encoding=None, max_cache_size=25, default_class=None, - variable_lookup='lenient', callback=None): + variable_lookup='strict', allow_exec=True, callback=None): """Create the template laoder. :param search_path: a list of absolute path names that should be searched for template files, or a string containing - a single absolute path + a single absolute path; alternatively, any item on + the list may be a ''load function'' that is passed + a filename and returns a file-like object and some + metadata :param auto_reload: whether to check the last modification time of template files, and reload them if they have changed :param default_encoding: the default encoding to assume when loading @@ -87,40 +95,48 @@ cache :param default_class: the default `Template` subclass to use when instantiating templates - :param variable_lookup: the variable lookup mechanism; either "lenient" - (the default), "strict", or a custom lookup + :param variable_lookup: the variable lookup mechanism; either "strict" + (the default), "lenient", or a custom lookup class + :param allow_exec: whether to allow Python code blocks in templates :param callback: (optional) a callback function that is invoked after a template was initialized by this loader; the function is passed the template object as only argument. This callback can be used for example to add any desired filters to the template :see: `LenientLookup`, `StrictLookup` + + :note: Changed in 0.5: Added the `allow_exec` argument """ from genshi.template.markup import MarkupTemplate self.search_path = search_path if self.search_path is None: self.search_path = [] - elif isinstance(self.search_path, basestring): + elif not isinstance(self.search_path, (list, tuple)): self.search_path = [self.search_path] + self.auto_reload = auto_reload + """Whether templates should be reloaded when the underlying file is + changed""" + self.default_encoding = default_encoding self.default_class = default_class or MarkupTemplate self.variable_lookup = variable_lookup - if callback is not None and not callable(callback): + self.allow_exec = allow_exec + if callback is not None and not hasattr(callback, '__call__'): raise TypeError('The "callback" parameter needs to be callable') self.callback = callback self._cache = LRUCache(max_cache_size) - self._mtime = {} - self._lock = threading.Lock() + self._uptodate = {} + self._lock = threading.RLock() def load(self, filename, relative_to=None, cls=None, encoding=None): """Load the template with the given name. - If the `filename` parameter is relative, this method searches the search - path trying to locate a template matching the given name. If the file - name is an absolute path, the search path is ignored. + If the `filename` parameter is relative, this method searches the + search path trying to locate a template matching the given name. If the + file name is an absolute path, the search path is ignored. If the requested template is not found, a `TemplateNotFound` exception is raised. Otherwise, a `Template` object is returned that represents @@ -143,29 +159,35 @@ :param encoding: the encoding of the template to load; defaults to the ``default_encoding`` of the loader instance :return: the loaded `Template` instance - :raises TemplateNotFound: if a template with the given name could not be - found + :raises TemplateNotFound: if a template with the given name could not + be found """ if cls is None: cls = self.default_class - if encoding is None: - encoding = self.default_encoding - if relative_to and not os.path.isabs(relative_to): + search_path = self.search_path + + # Make the filename relative to the template file its being loaded + # from, but only if that file is specified as a relative path, or no + # search path has been set up + if relative_to and (not search_path or not os.path.isabs(relative_to)): filename = os.path.join(os.path.dirname(relative_to), filename) + filename = os.path.normpath(filename) + cachekey = filename self._lock.acquire() try: # First check the cache to avoid reparsing the same file try: - tmpl = self._cache[filename] - if not self.auto_reload or \ - os.path.getmtime(tmpl.filepath) == self._mtime[filename]: + tmpl = self._cache[cachekey] + if not self.auto_reload: return tmpl - except KeyError: + uptodate = self._uptodate[cachekey] + if uptodate is not None and uptodate(): + return tmpl + except (KeyError, OSError): pass - search_path = self.search_path isabs = False if os.path.isabs(filename): @@ -178,40 +200,135 @@ # template is on the search path dirname = os.path.dirname(relative_to) if dirname not in search_path: - search_path = search_path + [dirname] + search_path = list(search_path) + [dirname] isabs = True elif not search_path: # Uh oh, don't know where to look for the template raise TemplateError('Search path for templates not configured') - for dirname in search_path: - filepath = os.path.join(dirname, filename) + for loadfunc in search_path: + if isinstance(loadfunc, basestring): + loadfunc = directory(loadfunc) try: - fileobj = open(filepath, 'U') + filepath, filename, fileobj, uptodate = loadfunc(filename) + except IOError: + continue + else: try: if isabs: # If the filename of either the included or the # including template is absolute, make sure the # included template gets an absolute path, too, - # so that nested include work properly without a + # so that nested includes work properly without a # search path - filename = os.path.join(dirname, filename) - dirname = '' - tmpl = cls(fileobj, basedir=dirname, filename=filename, - loader=self, lookup=self.variable_lookup, - encoding=encoding) + filename = filepath + tmpl = self._instantiate(cls, fileobj, filepath, + filename, encoding=encoding) if self.callback: self.callback(tmpl) - self._cache[filename] = tmpl - self._mtime[filename] = os.path.getmtime(filepath) + self._cache[cachekey] = tmpl + self._uptodate[cachekey] = uptodate finally: - fileobj.close() + if hasattr(fileobj, 'close'): + fileobj.close() return tmpl - except IOError: - continue raise TemplateNotFound(filename, search_path) finally: self._lock.release() + + def _instantiate(self, cls, fileobj, filepath, filename, encoding=None): + """Instantiate and return the `Template` object based on the given + class and parameters. + + This function is intended for subclasses to override if they need to + implement special template instantiation logic. Code that just uses + the `TemplateLoader` should use the `load` method instead. + + :param cls: the class of the template object to instantiate + :param fileobj: a readable file-like object containing the template + source + :param filepath: the absolute path to the template file + :param filename: the path to the template file relative to the search + path + :param encoding: the encoding of the template to load; defaults to the + ``default_encoding`` of the loader instance + :return: the loaded `Template` instance + :rtype: `Template` + """ + if encoding is None: + encoding = self.default_encoding + return cls(fileobj, filepath=filepath, filename=filename, loader=self, + encoding=encoding, lookup=self.variable_lookup, + allow_exec=self.allow_exec) + + def directory(path): + """Loader factory for loading templates from a local directory. + + :param path: the path to the local directory containing the templates + :return: the loader function to load templates from the given directory + :rtype: ``function`` + """ + def _load_from_directory(filename): + filepath = os.path.join(path, filename) + fileobj = open(filepath, 'U') + mtime = os.path.getmtime(filepath) + def _uptodate(): + return mtime == os.path.getmtime(filepath) + return filepath, filename, fileobj, _uptodate + return _load_from_directory + directory = staticmethod(directory) + + def package(name, path): + """Loader factory for loading templates from egg package data. + + :param name: the name of the package containing the resources + :param path: the path inside the package data + :return: the loader function to load templates from the given package + :rtype: ``function`` + """ + from pkg_resources import resource_stream + def _load_from_package(filename): + filepath = os.path.join(path, filename) + return filepath, filename, resource_stream(name, filepath), None + return _load_from_package + package = staticmethod(package) + + def prefixed(**delegates): + """Factory for a load function that delegates to other loaders + depending on the prefix of the requested template path. + + The prefix is stripped from the filename when passing on the load + request to the delegate. + + >>> load = prefixed( + ... app1 = lambda filename: ('app1', filename, None, None), + ... app2 = lambda filename: ('app2', filename, None, None) + ... ) + >>> print load('app1/foo.html') + ('app1', 'app1/foo.html', None, None) + >>> print load('app2/bar.html') + ('app2', 'app2/bar.html', None, None) + + :param delegates: mapping of path prefixes to loader functions + :return: the loader function + :rtype: ``function`` + """ + def _dispatch_by_prefix(filename): + for prefix, delegate in delegates.items(): + if filename.startswith(prefix): + if isinstance(delegate, basestring): + delegate = directory(delegate) + filepath, _, fileobj, uptodate = delegate( + filename[len(prefix):].lstrip('/\\') + ) + return filepath, filename, fileobj, uptodate + raise TemplateNotFound(filename, delegates.keys()) + return _dispatch_by_prefix + prefixed = staticmethod(prefixed) + +directory = TemplateLoader.directory +package = TemplateLoader.package +prefixed = TemplateLoader.prefixed
--- a/genshi/template/markup.py +++ b/genshi/template/markup.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -14,23 +14,17 @@ """Markup templating engine.""" from itertools import chain -import sys -from textwrap import dedent -from genshi.core import Attrs, Namespace, Stream, StreamEventKind +from genshi.core import Attrs, Markup, Namespace, Stream, StreamEventKind from genshi.core import START, END, START_NS, END_NS, TEXT, PI, COMMENT from genshi.input import XMLParser from genshi.template.base import BadDirectiveError, Template, \ TemplateSyntaxError, _apply_directives, \ - INCLUDE, SUB + EXEC, INCLUDE, SUB from genshi.template.eval import Suite from genshi.template.interpolation import interpolate from genshi.template.directives import * - -if sys.version_info < (2, 4): - _ctxt2dict = lambda ctxt: ctxt.frames[0] -else: - _ctxt2dict = lambda ctxt: ctxt +from genshi.template.text import NewTextTemplate __all__ = ['MarkupTemplate'] __docformat__ = 'restructuredtext en' @@ -47,11 +41,9 @@ <li>1</li><li>2</li><li>3</li> </ul> """ - EXEC = StreamEventKind('EXEC') - """Stream event kind representing a Python code suite to execute.""" - DIRECTIVE_NAMESPACE = Namespace('http://genshi.edgewall.org/') - XINCLUDE_NAMESPACE = Namespace('http://www.w3.org/2001/XInclude') + DIRECTIVE_NAMESPACE = 'http://genshi.edgewall.org/' + XINCLUDE_NAMESPACE = 'http://www.w3.org/2001/XInclude' directives = [('def', DefDirective), ('match', MatchDirective), @@ -65,140 +57,44 @@ ('content', ContentDirective), ('attrs', AttrsDirective), ('strip', StripDirective)] + serializer = 'xml' + _number_conv = Markup - def __init__(self, source, basedir=None, filename=None, loader=None, - encoding=None, lookup='lenient'): - Template.__init__(self, source, basedir=basedir, filename=filename, - loader=loader, encoding=encoding, lookup=lookup) + def __init__(self, source, filepath=None, filename=None, loader=None, + encoding=None, lookup='strict', allow_exec=True): + Template.__init__(self, source, filepath=filepath, filename=filename, + loader=loader, encoding=encoding, lookup=lookup, + allow_exec=allow_exec) + self.add_directives(self.DIRECTIVE_NAMESPACE, self) + + def _init_filters(self): + Template._init_filters(self) # Make sure the include filter comes after the match filter - if loader: + if self.loader: self.filters.remove(self._include) - self.filters += [self._exec, self._match] - if loader: + self.filters += [self._match] + if self.loader: self.filters.append(self._include) def _parse(self, source, encoding): - streams = [[]] # stacked lists of events of the "compiled" template - dirmap = {} # temporary mapping of directives to elements - ns_prefix = {} - depth = 0 - in_fallback = 0 - include_href = None - if not isinstance(source, Stream): source = XMLParser(source, filename=self.filename, encoding=encoding) + stream = [] for kind, data, pos in source: - stream = streams[-1] - if kind is START_NS: - # Strip out the namespace declaration for template directives - prefix, uri = data - ns_prefix[prefix] = uri - if uri not in (self.DIRECTIVE_NAMESPACE, - self.XINCLUDE_NAMESPACE): - stream.append((kind, data, pos)) - - elif kind is END_NS: - uri = ns_prefix.pop(data, None) - if uri and uri not in (self.DIRECTIVE_NAMESPACE, - self.XINCLUDE_NAMESPACE): + if kind is TEXT: + for kind, data, pos in interpolate(data, self.filepath, pos[1], + pos[2], lookup=self.lookup): stream.append((kind, data, pos)) - elif kind is START: - # Record any directive attributes in start tags - tag, attrs = data - directives = [] - strip = False - - if tag in self.DIRECTIVE_NAMESPACE: - cls = self._dir_by_name.get(tag.localname) - if cls is None: - raise BadDirectiveError(tag.localname, self.filepath, - pos[1]) - value = attrs.get(getattr(cls, 'ATTRIBUTE', None), '') - directives.append((cls, value, ns_prefix.copy(), pos)) - strip = True - - new_attrs = [] - for name, value in attrs: - if name in self.DIRECTIVE_NAMESPACE: - cls = self._dir_by_name.get(name.localname) - if cls is None: - raise BadDirectiveError(name.localname, - self.filepath, pos[1]) - directives.append((cls, value, ns_prefix.copy(), pos)) - else: - if value: - value = list(interpolate(value, self.basedir, - pos[0], pos[1], pos[2], - lookup=self.lookup)) - if len(value) == 1 and value[0][0] is TEXT: - value = value[0][1] - else: - value = [(TEXT, u'', pos)] - new_attrs.append((name, value)) - new_attrs = Attrs(new_attrs) - - if directives: - index = self._dir_order.index - directives.sort(lambda a, b: cmp(index(a[0]), index(b[0]))) - dirmap[(depth, tag)] = (directives, len(stream), strip) - - if tag in self.XINCLUDE_NAMESPACE: - if tag.localname == 'include': - include_href = new_attrs.get('href') - if not include_href: - raise TemplateSyntaxError('Include misses required ' - 'attribute "href"', - self.filepath, *pos[1:]) - streams.append([]) - elif tag.localname == 'fallback': - in_fallback += 1 - - else: - stream.append((kind, (tag, new_attrs), pos)) - - depth += 1 - - elif kind is END: - depth -= 1 - - if in_fallback and data == self.XINCLUDE_NAMESPACE['fallback']: - in_fallback -= 1 - elif data == self.XINCLUDE_NAMESPACE['include']: - fallback = streams.pop() - stream = streams[-1] - stream.append((INCLUDE, (include_href, fallback), pos)) - else: - stream.append((kind, data, pos)) - - # If there have have directive attributes with the corresponding - # start tag, move the events inbetween into a "subprogram" - if (depth, data) in dirmap: - directives, start_offset, strip = dirmap.pop((depth, data)) - substream = stream[start_offset:] - if strip: - substream = substream[1:-1] - stream[start_offset:] = [(SUB, (directives, substream), - pos)] - elif kind is PI and data[0] == 'python': + if not self.allow_exec: + raise TemplateSyntaxError('Python code blocks not allowed', + self.filepath, *pos[1:]) try: - # As Expat doesn't report whitespace between the PI target - # and the data, we have to jump through some hoops here to - # get correctly indented Python code - # Unfortunately, we'll still probably not get the line - # number quite right - lines = [line.expandtabs() for line in data[1].splitlines()] - first = lines[0] - rest = dedent('\n'.join(lines[1:])) - if first.rstrip().endswith(':') and not rest[0].isspace(): - rest = '\n'.join([' ' + line for line - in rest.splitlines()]) - source = '\n'.join([first, rest]) - suite = Suite(source, self.filepath, pos[1], + suite = Suite(data[1], self.filepath, pos[1], lookup=self.lookup) except SyntaxError, err: raise TemplateSyntaxError(err, self.filepath, @@ -206,12 +102,6 @@ pos[2] + (err.offset or 0)) stream.append((EXEC, suite, pos)) - elif kind is TEXT: - for kind, data, pos in interpolate(data, self.basedir, pos[0], - pos[1], pos[2], - lookup=self.lookup): - stream.append((kind, data, pos)) - elif kind is COMMENT: if not data.lstrip().startswith('!'): stream.append((kind, data, pos)) @@ -219,25 +109,206 @@ else: stream.append((kind, data, pos)) + return stream + + def _extract_directives(self, stream, namespace, factory): + depth = 0 + dirmap = {} # temporary mapping of directives to elements + new_stream = [] + ns_prefix = {} # namespace prefixes in use + + for kind, data, pos in stream: + + if kind is START: + tag, attrs = data + directives = [] + strip = False + + if tag.namespace == namespace: + cls = factory.get_directive(tag.localname) + if cls is None: + raise BadDirectiveError(tag.localname, + self.filepath, pos[1]) + args = dict([(name.localname, value) for name, value + in attrs if not name.namespace]) + directives.append((cls, args, ns_prefix.copy(), pos)) + strip = True + + new_attrs = [] + for name, value in attrs: + if name.namespace == namespace: + cls = factory.get_directive(name.localname) + if cls is None: + raise BadDirectiveError(name.localname, + self.filepath, pos[1]) + if type(value) is list and len(value) == 1: + value = value[0][1] + directives.append((cls, value, ns_prefix.copy(), + pos)) + else: + new_attrs.append((name, value)) + new_attrs = Attrs(new_attrs) + + if directives: + directives.sort(self.compare_directives()) + dirmap[(depth, tag)] = (directives, len(new_stream), + strip) + + new_stream.append((kind, (tag, new_attrs), pos)) + depth += 1 + + elif kind is END: + depth -= 1 + new_stream.append((kind, data, pos)) + + # If there have have directive attributes with the + # corresponding start tag, move the events inbetween into + # a "subprogram" + if (depth, data) in dirmap: + directives, offset, strip = dirmap.pop((depth, data)) + substream = new_stream[offset:] + if strip: + substream = substream[1:-1] + new_stream[offset:] = [ + (SUB, (directives, substream), pos) + ] + + elif kind is SUB: + directives, substream = data + substream = self._extract_directives(substream, namespace, + factory) + + if len(substream) == 1 and substream[0][0] is SUB: + added_directives, substream = substream[0][1] + directives += added_directives + + new_stream.append((kind, (directives, substream), pos)) + + elif kind is START_NS: + # Strip out the namespace declaration for template + # directives + prefix, uri = data + ns_prefix[prefix] = uri + if uri != namespace: + new_stream.append((kind, data, pos)) + + elif kind is END_NS: + uri = ns_prefix.pop(data, None) + if uri and uri != namespace: + new_stream.append((kind, data, pos)) + + else: + new_stream.append((kind, data, pos)) + + return new_stream + + def _extract_includes(self, stream): + streams = [[]] # stacked lists of events of the "compiled" template + prefixes = {} + fallbacks = [] + includes = [] + xinclude_ns = Namespace(self.XINCLUDE_NAMESPACE) + + for kind, data, pos in stream: + stream = streams[-1] + + if kind is START: + # Record any directive attributes in start tags + tag, attrs = data + if tag in xinclude_ns: + if tag.localname == 'include': + include_href = attrs.get('href') + if not include_href: + raise TemplateSyntaxError('Include misses required ' + 'attribute "href"', + self.filepath, *pos[1:]) + includes.append((include_href, attrs.get('parse'))) + streams.append([]) + elif tag.localname == 'fallback': + streams.append([]) + fallbacks.append(streams[-1]) + else: + stream.append((kind, (tag, attrs), pos)) + + elif kind is END: + if fallbacks and data == xinclude_ns['fallback']: + assert streams.pop() is fallbacks[-1] + elif data == xinclude_ns['include']: + fallback = None + if len(fallbacks) == len(includes): + fallback = fallbacks.pop() + streams.pop() # discard anything between the include tags + # and the fallback element + stream = streams[-1] + href, parse = includes.pop() + try: + cls = { + 'xml': MarkupTemplate, + 'text': NewTextTemplate + }[parse or 'xml'] + except KeyError: + raise TemplateSyntaxError('Invalid value for "parse" ' + 'attribute of include', + self.filepath, *pos[1:]) + stream.append((INCLUDE, (href, cls, fallback), pos)) + else: + stream.append((kind, data, pos)) + + elif kind is START_NS and data[1] == xinclude_ns: + # Strip out the XInclude namespace + prefixes[data[0]] = data[1] + + elif kind is END_NS and data in prefixes: + prefixes.pop(data) + + else: + stream.append((kind, data, pos)) + assert len(streams) == 1 return streams[0] - def _exec(self, stream, ctxt): - """Internal stream filter that executes code in ``<?python ?>`` - processing instructions. + def _interpolate_attrs(self, stream): + for kind, data, pos in stream: + + if kind is START: + # Record any directive attributes in start tags + tag, attrs = data + new_attrs = [] + for name, value in attrs: + if value: + value = list(interpolate(value, self.filepath, pos[1], + pos[2], lookup=self.lookup)) + if len(value) == 1 and value[0][0] is TEXT: + value = value[0][1] + new_attrs.append((name, value)) + data = tag, Attrs(new_attrs) + + yield kind, data, pos + + def _prepare(self, stream): + return Template._prepare(self, + self._extract_includes(self._interpolate_attrs(stream)) + ) + + def add_directives(self, namespace, factory): + """Register a custom `DirectiveFactory` for a given namespace. + + :param namespace: the namespace URI + :type namespace: `basestring` + :param factory: the directive factory to register + :type factory: `DirectiveFactory` + :since: version 0.6 """ - for event in stream: - if event[0] is EXEC: - event[1].execute(_ctxt2dict(ctxt)) - else: - yield event + assert not self._prepared, 'Too late for adding directives, ' \ + 'template already prepared' + self._stream = self._extract_directives(self._stream, namespace, + factory) - def _match(self, stream, ctxt, match_templates=None): + def _match(self, stream, ctxt, start=0, end=None, **vars): """Internal stream filter that applies any defined match templates to the stream. """ - if match_templates is None: - match_templates = ctxt._match_templates + match_templates = ctxt._match_templates tail = [] def _strip(stream): @@ -263,10 +334,15 @@ yield event continue - for idx, (test, path, template, namespaces, directives) in \ - enumerate(match_templates): + for idx, (test, path, template, hints, namespaces, directives) \ + in enumerate(match_templates): + if idx < start or end is not None and idx >= end: + continue if test(event, namespaces, ctxt) is True: + if 'match_once' in hints: + del match_templates[idx] + idx -= 1 # Let the remaining match templates know about the event so # they get a chance to update their internal state @@ -275,35 +351,46 @@ # Consume and store all events until an end event # corresponding to this start event is encountered - content = chain([event], - self._match(_strip(stream), ctxt, - [match_templates[idx]]), - tail) - content = list(self._include(content, ctxt)) - - for test in [mt[0] for mt in match_templates]: - test(tail[0], namespaces, ctxt, updateonly=True) + pre_end = idx + 1 + if 'match_once' not in hints and 'not_recursive' in hints: + pre_end -= 1 + inner = _strip(stream) + if pre_end > 0: + inner = self._match(inner, ctxt, end=pre_end) + content = self._include(chain([event], inner, tail), ctxt) + if 'not_buffered' not in hints: + content = list(content) # Make the select() function available in the body of the # match template + selected = [False] def select(path): + selected[0] = True return Stream(content).select(path, namespaces, ctxt) - ctxt.push(dict(select=select)) + vars = dict(select=select) # Recursively process the output - template = _apply_directives(template, ctxt, directives) - for event in self._match(self._eval(self._flatten(template, - ctxt), - ctxt), ctxt, - match_templates[:idx] + - match_templates[idx + 1:]): + template = _apply_directives(template, directives, ctxt, + **vars) + for event in self._match(self._flatten(template, ctxt, + **vars), + ctxt, start=idx + 1, **vars): yield event - ctxt.pop() + # If the match template did not actually call select to + # consume the matched stream, the original events need to + # be consumed here or they'll get appended to the output + if not selected[0]: + for event in content: + pass + + # Let the remaining match templates know about the last + # event in the matched content, so they can update their + # internal state accordingly + for test in [mt[0] for mt in match_templates]: + test(tail[0], namespaces, ctxt, updateonly=True) + break else: # no matches yield event - - -EXEC = MarkupTemplate.EXEC
--- a/genshi/template/plugin.py +++ b/genshi/template/plugin.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # Copyright (C) 2006 Matthew Good # All rights reserved. # @@ -16,14 +16,12 @@ CherryPy/Buffet. """ -from pkg_resources import resource_filename - from genshi.input import ET, HTML, XML from genshi.output import DocType from genshi.template.base import Template from genshi.template.loader import TemplateLoader from genshi.template.markup import MarkupTemplate -from genshi.template.text import TextTemplate +from genshi.template.text import TextTemplate, NewTextTemplate __all__ = ['ConfigurationError', 'AbstractTemplateEnginePlugin', 'MarkupTemplateEnginePlugin', 'TextTemplateEnginePlugin'] @@ -58,16 +56,28 @@ raise ConfigurationError('Invalid value for max_cache_size: "%s"' % options.get('genshi.max_cache_size')) - lookup_errors = options.get('genshi.lookup_errors', 'lenient') + loader_callback = options.get('genshi.loader_callback', None) + if loader_callback and not hasattr(loader_callback, '__call__'): + raise ConfigurationError('loader callback must be a function') + + lookup_errors = options.get('genshi.lookup_errors', 'strict') if lookup_errors not in ('lenient', 'strict'): raise ConfigurationError('Unknown lookup errors mode "%s"' % lookup_errors) + try: + allow_exec = bool(options.get('genshi.allow_exec', True)) + except ValueError: + raise ConfigurationError('Invalid value for allow_exec "%s"' % + options.get('genshi.allow_exec')) + self.loader = TemplateLoader(filter(None, search_path), auto_reload=auto_reload, max_cache_size=max_cache_size, default_class=self.template_class, - variable_lookup=lookup_errors) + variable_lookup=lookup_errors, + allow_exec=allow_exec, + callback=loader_callback) def load_template(self, templatename, template_string=None): """Find a template specified in python 'dot' notation, or load one from @@ -79,13 +89,14 @@ if self.use_package_naming: divider = templatename.rfind('.') if divider >= 0: + from pkg_resources import resource_filename package = templatename[:divider] basename = templatename[divider + 1:] + self.extension templatename = resource_filename(package, basename) return self.loader.load(templatename) - def _get_render_options(self, format=None): + def _get_render_options(self, format=None, fragment=False): if format is None: format = self.default_format kwargs = {'method': format} @@ -95,7 +106,7 @@ def render(self, info, format=None, fragment=False, template=None): """Render the template to a string using the provided info.""" - kwargs = self._get_render_options(format=format) + kwargs = self._get_render_options(format=format, fragment=fragment) return self.transform(info, template).render(**kwargs) def transform(self, info, template): @@ -128,10 +139,10 @@ raise ConfigurationError('Unknown output format %r' % format) self.default_format = format - def _get_render_options(self, format=None): + def _get_render_options(self, format=None, fragment=False): kwargs = super(MarkupTemplateEnginePlugin, - self)._get_render_options(format) - if self.default_doctype: + self)._get_render_options(format, fragment) + if self.default_doctype and not fragment: kwargs['doctype'] = self.default_doctype return kwargs @@ -150,3 +161,15 @@ template_class = TextTemplate extension = '.txt' default_format = 'text' + + def __init__(self, extra_vars_func=None, options=None): + if options is None: + options = {} + + new_syntax = options.get('genshi.new_text_syntax') + if isinstance(new_syntax, basestring): + new_syntax = new_syntax.lower() in ('1', 'on', 'yes', 'true') + if new_syntax: + self.template_class = NewTextTemplate + + AbstractTemplateEnginePlugin.__init__(self, extra_vars_func, options)
--- a/genshi/template/tests/directives.py +++ b/genshi/template/tests/directives.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -382,6 +382,7 @@ """) self.assertEqual(""" Hi, you! + """, str(tmpl.generate())) def test_function_with_star_args(self): @@ -482,9 +483,35 @@ try: list(tmpl.generate(foo=12)) self.fail('Expected TemplateRuntimeError') - except TemplateRuntimeError, e: + except TypeError, e: + assert (str(e) == "iteration over non-sequence" or + str(e) == "'int' object is not iterable") + exc_type, exc_value, exc_traceback = sys.exc_info() + frame = exc_traceback.tb_next + frames = [] + while frame.tb_next: + frame = frame.tb_next + frames.append(frame) + self.assertEqual("<Expression u'iter(foo)'>", + frames[-1].tb_frame.f_code.co_name) + self.assertEqual('test.html', + frames[-1].tb_frame.f_code.co_filename) + self.assertEqual(2, frames[-1].tb_lineno) + + def test_for_with_empty_value(self): + """ + Verify an empty 'for' value is an error + """ + try: + MarkupTemplate("""<doc xmlns:py="http://genshi.edgewall.org/"> + <py:for each=""> + empty + </py:for> + </doc>""", filename='test.html').generate() + self.fail('ExpectedTemplateSyntaxError') + except TemplateSyntaxError, e: self.assertEqual('test.html', e.filename) - if sys.version_info[:2] >= (2, 4): + if sys.version_info[:2] > (2,4): self.assertEqual(2, e.lineno) @@ -620,6 +647,32 @@ </body> </html>""", str(tmpl.generate())) + def test_recursive_match_3(self): + tmpl = MarkupTemplate("""<test xmlns:py="http://genshi.edgewall.org/"> + <py:match path="b[@type='bullet']"> + <bullet>${select('*|text()')}</bullet> + </py:match> + <py:match path="group[@type='bullet']"> + <ul>${select('*')}</ul> + </py:match> + <py:match path="b"> + <generic>${select('*|text()')}</generic> + </py:match> + + <b> + <group type="bullet"> + <b type="bullet">1</b> + <b type="bullet">2</b> + </group> + </b> + </test> + """) + self.assertEqual("""<test> + <generic> + <ul><bullet>1</bullet><bullet>2</bullet></ul> + </generic> + </test>""", str(tmpl.generate())) + def test_not_match_self(self): """ See http://genshi.edgewall.org/ticket/77 @@ -842,6 +895,54 @@ </body> </html>""", str(tmpl.generate())) + def test_match_with_once_attribute(self): + tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/"> + <py:match path="body" once="true"><body> + <div id="wrap"> + ${select("*")} + </div> + </body></py:match> + <body> + <p>Foo</p> + </body> + <body> + <p>Bar</p> + </body> + </html>""") + self.assertEqual("""<html> + <body> + <div id="wrap"> + <p>Foo</p> + </div> + </body> + <body> + <p>Bar</p> + </body> + </html>""", str(tmpl.generate())) + + def test_match_with_recursive_attribute(self): + tmpl = MarkupTemplate("""<doc xmlns:py="http://genshi.edgewall.org/"> + <py:match path="elem" recursive="false"><elem> + <div class="elem"> + ${select('*')} + </div> + </elem></py:match> + <elem> + <subelem> + <elem/> + </subelem> + </elem> + </doc>""") + self.assertEqual("""<doc> + <elem> + <div class="elem"> + <subelem> + <elem/> + </subelem> + </div> + </elem> + </doc>""", str(tmpl.generate())) + # FIXME #def test_match_after_step(self): # tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/"> @@ -857,6 +958,20 @@ # </div>""", str(tmpl.generate())) +class ContentDirectiveTestCase(unittest.TestCase): + """Tests for the `py:content` template directive.""" + + def test_as_element(self): + try: + MarkupTemplate("""<doc xmlns:py="http://genshi.edgewall.org/"> + <py:content foo="">Foo</py:content> + </doc>""", filename='test.html').generate() + self.fail('Expected TemplateSyntaxError') + except TemplateSyntaxError, e: + self.assertEqual('test.html', e.filename) + self.assertEqual(2, e.lineno) + + class ReplaceDirectiveTestCase(unittest.TestCase): """Tests for the `py:replace` template directive.""" @@ -866,14 +981,21 @@ expression is supplied. """ try: - tmpl = MarkupTemplate("""<doc xmlns:py="http://genshi.edgewall.org/"> + MarkupTemplate("""<doc xmlns:py="http://genshi.edgewall.org/"> <elem py:replace="">Foo</elem> - </doc>""", filename='test.html') + </doc>""", filename='test.html').generate() self.fail('Expected TemplateSyntaxError') except TemplateSyntaxError, e: self.assertEqual('test.html', e.filename) - if sys.version_info[:2] >= (2, 4): - self.assertEqual(2, e.lineno) + self.assertEqual(2, e.lineno) + + def test_as_element(self): + tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/"> + <py:replace value="title" /> + </div>""", filename='test.html') + self.assertEqual("""<div> + Test + </div>""", str(tmpl.generate(title='Test'))) class StripDirectiveTestCase(unittest.TestCase): @@ -968,6 +1090,22 @@ here are two semicolons: ;; </div>""", str(tmpl.generate())) + def test_ast_transformation(self): + """ + Verify that the usual template expression AST transformations are + applied despite the code being compiled to a `Suite` object. + """ + tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/"> + <span py:with="bar=foo.bar"> + $bar + </span> + </div>""") + self.assertEqual("""<div> + <span> + 42 + </span> + </div>""", str(tmpl.generate(foo={'bar': 42}))) + def test_unicode_expr(self): tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/"> <span py:with="weeks=(u'一', u'二', u'三', u'四', u'五', u'六', u'日')"> @@ -979,6 +1117,16 @@ 一二三四五六日 </span> </div>""", str(tmpl.generate())) + + def test_with_empty_value(self): + """ + Verify that an empty py:with works (useless, but legal) + """ + tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/"> + <span py:with="">Text</span></div>""") + + self.assertEqual("""<div> + <span>Text</span></div>""", str(tmpl.generate())) def suite(): @@ -990,6 +1138,7 @@ suite.addTest(unittest.makeSuite(ForDirectiveTestCase, 'test')) suite.addTest(unittest.makeSuite(IfDirectiveTestCase, 'test')) suite.addTest(unittest.makeSuite(MatchDirectiveTestCase, 'test')) + suite.addTest(unittest.makeSuite(ContentDirectiveTestCase, 'test')) suite.addTest(unittest.makeSuite(ReplaceDirectiveTestCase, 'test')) suite.addTest(unittest.makeSuite(StripDirectiveTestCase, 'test')) suite.addTest(unittest.makeSuite(WithDirectiveTestCase, 'test'))
--- a/genshi/template/tests/eval.py +++ b/genshi/template/tests/eval.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -12,10 +12,13 @@ # history and logs, available at http://genshi.edgewall.org/log/. import doctest +import pickle +from StringIO import StringIO import sys import unittest from genshi.core import Markup +from genshi.template.base import Context from genshi.template.eval import Expression, Suite, Undefined, UndefinedError, \ UNDEFINED @@ -32,6 +35,14 @@ self.assertEqual(hash(expr), hash(Expression('x,y'))) self.assertNotEqual(hash(expr), hash(Expression('y, x'))) + def test_pickle(self): + expr = Expression('1 < 2') + buf = StringIO() + pickle.dump(expr, buf, 2) + buf.seek(0) + unpickled = pickle.load(buf) + assert unpickled.evaluate({}) is True + def test_name_lookup(self): self.assertEqual('bar', Expression('foo').evaluate({'foo': 'bar'})) self.assertEqual(id, Expression('id').evaluate({})) @@ -184,8 +195,9 @@ def test_compare_ne(self): self.assertEqual(False, Expression("1 != 1").evaluate({})) self.assertEqual(False, Expression("x != y").evaluate({'x': 1, 'y': 1})) - self.assertEqual(False, Expression("1 <> 1").evaluate({})) - self.assertEqual(False, Expression("x <> y").evaluate({'x': 1, 'y': 1})) + if sys.version < '3': + self.assertEqual(False, Expression("1 <> 1").evaluate({})) + self.assertEqual(False, Expression("x <> y").evaluate({'x': 1, 'y': 1})) def test_compare_lt(self): self.assertEqual(True, Expression("1 < 2").evaluate({})) @@ -230,16 +242,9 @@ self.assertEqual(42, expr.evaluate({'foo': foo, 'bar': {"x": 42}})) def test_lambda(self): - # Define a custom `sorted` function cause the builtin isn't available - # on Python 2.3 - def sorted(items, compfunc): - items.sort(compfunc) - return items - data = {'items': [{'name': 'b', 'value': 0}, {'name': 'a', 'value': 1}], - 'sorted': sorted} - expr = Expression("sorted(items, lambda a, b: cmp(a.name, b.name))") - self.assertEqual([{'name': 'a', 'value': 1}, {'name': 'b', 'value': 0}], - expr.evaluate(data)) + data = {'items': range(5)} + expr = Expression("filter(lambda x: x > 2, items)") + self.assertEqual([3, 4], expr.evaluate(data)) def test_list_comprehension(self): expr = Expression("[n for n in numbers if n < 2]") @@ -263,30 +268,27 @@ expr = Expression("[i['name'] for i in items if i['value'] > 1]") self.assertEqual(['b'], expr.evaluate({'items': items})) - if sys.version_info >= (2, 4): - # Generator expressions only supported in Python 2.4 and up - - def test_generator_expression(self): - expr = Expression("list(n for n in numbers if n < 2)") - self.assertEqual([0, 1], expr.evaluate({'numbers': range(5)})) - - expr = Expression("list((i, n + 1) for i, n in enumerate(numbers))") - self.assertEqual([(0, 1), (1, 2), (2, 3), (3, 4), (4, 5)], - expr.evaluate({'numbers': range(5)})) + def test_generator_expression(self): + expr = Expression("list(n for n in numbers if n < 2)") + self.assertEqual([0, 1], expr.evaluate({'numbers': range(5)})) - expr = Expression("list(offset + n for n in numbers)") - self.assertEqual([2, 3, 4, 5, 6], - expr.evaluate({'numbers': range(5), 'offset': 2})) + expr = Expression("list((i, n + 1) for i, n in enumerate(numbers))") + self.assertEqual([(0, 1), (1, 2), (2, 3), (3, 4), (4, 5)], + expr.evaluate({'numbers': range(5)})) - def test_generator_expression_with_getattr(self): - items = [{'name': 'a', 'value': 1}, {'name': 'b', 'value': 2}] - expr = Expression("list(i.name for i in items if i.value > 1)") - self.assertEqual(['b'], expr.evaluate({'items': items})) + expr = Expression("list(offset + n for n in numbers)") + self.assertEqual([2, 3, 4, 5, 6], + expr.evaluate({'numbers': range(5), 'offset': 2})) - def test_generator_expression_with_getitem(self): - items = [{'name': 'a', 'value': 1}, {'name': 'b', 'value': 2}] - expr = Expression("list(i['name'] for i in items if i['value'] > 1)") - self.assertEqual(['b'], expr.evaluate({'items': items})) + def test_generator_expression_with_getattr(self): + items = [{'name': 'a', 'value': 1}, {'name': 'b', 'value': 2}] + expr = Expression("list(i.name for i in items if i.value > 1)") + self.assertEqual(['b'], expr.evaluate({'items': items})) + + def test_generator_expression_with_getitem(self): + items = [{'name': 'a', 'value': 1}, {'name': 'b', 'value': 2}] + expr = Expression("list(i['name'] for i in items if i['value'] > 1)") + self.assertEqual(['b'], expr.evaluate({'items': items})) if sys.version_info >= (2, 5): def test_conditional_expression(self): @@ -321,7 +323,8 @@ self.assertEqual([0, 1, 2, 3], expr.evaluate({'numbers': range(5)})) def test_access_undefined(self): - expr = Expression("nothing", filename='index.html', lineno=50) + expr = Expression("nothing", filename='index.html', lineno=50, + lookup='lenient') retval = expr.evaluate({}) assert isinstance(retval, Undefined) self.assertEqual('nothing', retval._name) @@ -332,23 +335,45 @@ def __repr__(self): return '<Something>' something = Something() - expr = Expression('something.nil', filename='index.html', lineno=50) + expr = Expression('something.nil', filename='index.html', lineno=50, + lookup='lenient') retval = expr.evaluate({'something': something}) assert isinstance(retval, Undefined) self.assertEqual('nil', retval._name) assert retval._owner is something + def test_getattr_exception(self): + class Something(object): + def prop_a(self): + raise NotImplementedError + prop_a = property(prop_a) + def prop_b(self): + raise AttributeError + prop_b = property(prop_b) + self.assertRaises(NotImplementedError, + Expression('s.prop_a').evaluate, {'s': Something()}) + self.assertRaises(AttributeError, + Expression('s.prop_b').evaluate, {'s': Something()}) + def test_getitem_undefined_string(self): class Something(object): def __repr__(self): return '<Something>' something = Something() - expr = Expression('something["nil"]', filename='index.html', lineno=50) + expr = Expression('something["nil"]', filename='index.html', lineno=50, + lookup='lenient') retval = expr.evaluate({'something': something}) assert isinstance(retval, Undefined) self.assertEqual('nil', retval._name) assert retval._owner is something + def test_getitem_exception(self): + class Something(object): + def __getitem__(self, key): + raise NotImplementedError + self.assertRaises(NotImplementedError, + Expression('s["foo"]').evaluate, {'s': Something()}) + def test_error_access_undefined(self): expr = Expression("nothing", filename='index.html', lineno=50, lookup='strict') @@ -420,6 +445,29 @@ class SuiteTestCase(unittest.TestCase): + def test_pickle(self): + suite = Suite('foo = 42') + buf = StringIO() + pickle.dump(suite, buf, 2) + buf.seek(0) + unpickled = pickle.load(buf) + data = {} + unpickled.execute(data) + self.assertEqual(42, data['foo']) + + def test_internal_shadowing(self): + # The context itself is stored in the global execution scope of a suite + # It used to get stored under the name 'data', which meant the + # following test would fail, as the user defined 'data' variable + # shadowed the Genshi one. We now use the name '__data__' to avoid + # conflicts + suite = Suite("""data = [] +bar = foo +""") + data = {'foo': 42} + suite.execute(data) + self.assertEqual(42, data['bar']) + def test_assign(self): suite = Suite("foo = 42") data = {} @@ -434,7 +482,8 @@ self.assertEqual(None, data['donothing']()) def test_def_with_multiple_statements(self): - suite = Suite("""def donothing(): + suite = Suite(""" +def donothing(): if True: return foo """) @@ -443,6 +492,84 @@ assert 'donothing' in data self.assertEqual('bar', data['donothing']()) + def test_def_using_nonlocal(self): + suite = Suite(""" +values = [] +def add(value): + if value not in values: + values.append(value) +add('foo') +add('bar') +""") + data = {} + suite.execute(data) + self.assertEqual(['foo', 'bar'], data['values']) + + def test_def_some_defaults(self): + suite = Suite(""" +def difference(v1, v2=10): + return v1 - v2 +x = difference(20, 19) +y = difference(20) +""") + data = {} + suite.execute(data) + self.assertEqual(1, data['x']) + self.assertEqual(10, data['y']) + + def test_def_all_defaults(self): + suite = Suite(""" +def difference(v1=100, v2=10): + return v1 - v2 +x = difference(20, 19) +y = difference(20) +z = difference() +""") + data = {} + suite.execute(data) + self.assertEqual(1, data['x']) + self.assertEqual(10, data['y']) + self.assertEqual(90, data['z']) + + def test_def_vararg(self): + suite = Suite(""" +def mysum(*others): + rv = 0 + for n in others: + rv = rv + n + return rv +x = mysum(1, 2, 3) +""") + data = {} + suite.execute(data) + self.assertEqual(6, data['x']) + + def test_def_kwargs(self): + suite = Suite(""" +def smash(**kw): + return [''.join(i) for i in kw.items()] +x = smash(foo='abc', bar='def') +""") + data = {} + suite.execute(data) + self.assertEqual(['fooabc', 'bardef'], data['x']) + + def test_def_nested(self): + suite = Suite(""" +def doit(): + values = [] + def add(value): + if value not in values: + values.append(value) + add('foo') + add('bar') + return values +x = doit() +""") + data = {} + suite.execute(data) + self.assertEqual(['foo', 'bar'], data['x']) + def test_delete(self): suite = Suite("""foo = 42 del foo @@ -457,12 +584,50 @@ suite.execute(data) assert 'plain' in data + def test_class_in_def(self): + suite = Suite(""" +def create(): + class Foobar(object): + def __str__(self): + return 'foobar' + return Foobar() +x = create() +""") + data = {} + suite.execute(data) + self.assertEqual('foobar', str(data['x'])) + + def test_class_with_methods(self): + suite = Suite("""class plain(object): + def donothing(): + pass +""") + data = {} + suite.execute(data) + assert 'plain' in data + def test_import(self): suite = Suite("from itertools import ifilter") data = {} suite.execute(data) assert 'ifilter' in data + def test_import_star(self): + suite = Suite("from itertools import *") + data = Context() + suite.execute(data) + assert 'ifilter' in data + + def test_import_in_def(self): + suite = Suite("""def fun(): + from itertools import ifilter + return ifilter(None, xrange(3)) +""") + data = Context() + suite.execute(data) + assert 'ifilter' not in data + self.assertEqual([1, 2], list(data['fun']())) + def test_for(self): suite = Suite("""x = [] for i in range(3): @@ -472,6 +637,18 @@ suite.execute(data) self.assertEqual([0, 1, 4], data['x']) + def test_for_in_def(self): + suite = Suite("""def loop(): + for i in range(10): + if i == 5: + break + return i +""") + data = {} + suite.execute(data) + assert 'loop' in data + self.assertEqual(5, data['loop']()) + def test_if(self): suite = Suite("""if foo == 42: x = True @@ -525,6 +702,25 @@ def test_local_augmented_assign(self): Suite("x = 1; x += 42; assert x == 43").execute({}) + def test_augmented_assign_in_def(self): + d = {} + Suite("""def foo(): + i = 1 + i += 1 + return i +x = foo()""").execute(d) + self.assertEqual(2, d['x']) + + def test_augmented_assign_in_loop_in_def(self): + d = {} + Suite("""def foo(): + i = 0 + for n in range(5): + i += n + return i +x = foo()""").execute(d) + self.assertEqual(10, d['x']) + def test_assign_in_list(self): suite = Suite("[d['k']] = 'foo',; assert d['k'] == 'foo'") d = {"k": "bar"} @@ -569,7 +765,7 @@ def test_delitem(self): d = {'k': 'foo'} Suite("del d['k']").execute({'d': d}) - self.failIf('k' in d, `d`) + self.failIf('k' in d, repr(d)) def suite():
--- a/genshi/template/tests/interpolation.py +++ b/genshi/template/tests/interpolation.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2007 Edgewall Software +# Copyright (C) 2007-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -60,6 +60,12 @@ self.assertEqual(TEXT, parts[0][0]) self.assertEqual('$bla', parts[0][1]) + def test_interpolate_short_escaped_2(self): + parts = list(interpolate('my $$bla = 2')) + self.assertEqual(1, len(parts)) + self.assertEqual(TEXT, parts[0][0]) + self.assertEqual('my $bla = 2', parts[0][1]) + def test_interpolate_short_doubleescaped(self): parts = list(interpolate('$$$bla')) self.assertEqual(2, len(parts)) @@ -180,6 +186,11 @@ self.assertEqual(TEXT, parts[2][0]) self.assertEqual(' baz', parts[2][1]) + def test_interpolate_triplequoted(self): + parts = list(interpolate('${"""foo\nbar"""}')) + self.assertEqual(1, len(parts)) + self.assertEqual('"""foo\nbar"""', parts[0][1].source) + def suite(): suite = unittest.TestSuite()
--- a/genshi/template/tests/loader.py +++ b/genshi/template/tests/loader.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -104,6 +104,34 @@ <div>Included</div> </html>""", tmpl.generate().render()) + def test_relative_include_samesubdir(self): + file1 = open(os.path.join(self.dirname, 'tmpl1.html'), 'w') + try: + file1.write("""<div>Included tmpl1.html</div>""") + finally: + file1.close() + + os.mkdir(os.path.join(self.dirname, 'sub')) + file2 = open(os.path.join(self.dirname, 'sub', 'tmpl1.html'), 'w') + try: + file2.write("""<div>Included sub/tmpl1.html</div>""") + finally: + file2.close() + + file3 = open(os.path.join(self.dirname, 'sub', 'tmpl2.html'), 'w') + try: + file3.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="tmpl1.html" /> + </html>""") + finally: + file3.close() + + loader = TemplateLoader([self.dirname]) + tmpl = loader.load('sub/tmpl2.html') + self.assertEqual("""<html> + <div>Included sub/tmpl1.html</div> + </html>""", tmpl.generate().render()) + def test_relative_include_without_search_path(self): file1 = open(os.path.join(self.dirname, 'tmpl1.html'), 'w') try: @@ -172,6 +200,108 @@ <div>Included</div> </html>""", tmpl2.generate().render()) + def test_relative_absolute_template_preferred(self): + file1 = open(os.path.join(self.dirname, 'tmpl1.html'), 'w') + try: + file1.write("""<div>Included</div>""") + finally: + file1.close() + + os.mkdir(os.path.join(self.dirname, 'sub')) + file2 = open(os.path.join(self.dirname, 'sub', 'tmpl1.html'), 'w') + try: + file2.write("""<div>Included from sub</div>""") + finally: + file2.close() + + file3 = open(os.path.join(self.dirname, 'sub', 'tmpl2.html'), 'w') + try: + file3.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="tmpl1.html" /> + </html>""") + finally: + file3.close() + + loader = TemplateLoader() + tmpl = loader.load(os.path.abspath(os.path.join(self.dirname, 'sub', + 'tmpl2.html'))) + self.assertEqual("""<html> + <div>Included from sub</div> + </html>""", tmpl.generate().render()) + + def test_abspath_caching(self): + abspath = os.path.join(self.dirname, 'abs') + os.mkdir(abspath) + file1 = open(os.path.join(abspath, 'tmpl1.html'), 'w') + try: + file1.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="tmpl2.html" /> + </html>""") + finally: + file1.close() + + file2 = open(os.path.join(abspath, 'tmpl2.html'), 'w') + try: + file2.write("""<div>Included from abspath.</div>""") + finally: + file2.close() + + searchpath = os.path.join(self.dirname, 'searchpath') + os.mkdir(searchpath) + file3 = open(os.path.join(searchpath, 'tmpl2.html'), 'w') + try: + file3.write("""<div>Included from searchpath.</div>""") + finally: + file3.close() + + loader = TemplateLoader(searchpath) + tmpl1 = loader.load(os.path.join(abspath, 'tmpl1.html')) + self.assertEqual("""<html> + <div>Included from searchpath.</div> + </html>""", tmpl1.generate().render()) + assert 'tmpl2.html' in loader._cache + + def test_abspath_include_caching_without_search_path(self): + file1 = open(os.path.join(self.dirname, 'tmpl1.html'), 'w') + try: + file1.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="tmpl2.html" /> + </html>""") + finally: + file1.close() + + file2 = open(os.path.join(self.dirname, 'tmpl2.html'), 'w') + try: + file2.write("""<div>Included</div>""") + finally: + file2.close() + + os.mkdir(os.path.join(self.dirname, 'sub')) + file3 = open(os.path.join(self.dirname, 'sub', 'tmpl1.html'), 'w') + try: + file3.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="tmpl2.html" /> + </html>""") + finally: + file3.close() + + file4 = open(os.path.join(self.dirname, 'sub', 'tmpl2.html'), 'w') + try: + file4.write("""<div>Included from sub</div>""") + finally: + file4.close() + + loader = TemplateLoader() + tmpl1 = loader.load(os.path.join(self.dirname, 'tmpl1.html')) + self.assertEqual("""<html> + <div>Included</div> + </html>""", tmpl1.generate().render()) + tmpl2 = loader.load(os.path.join(self.dirname, 'sub', 'tmpl1.html')) + self.assertEqual("""<html> + <div>Included from sub</div> + </html>""", tmpl2.generate().render()) + assert 'tmpl2.html' not in loader._cache + def test_load_with_default_encoding(self): f = open(os.path.join(self.dirname, 'tmpl.html'), 'w') try: @@ -219,6 +349,108 @@ <p>Hello, hello</p> </html>""", tmpl.generate().render()) + def test_prefix_delegation_to_directories(self): + """ + Test prefix delegation with the following layout: + + templates/foo.html + sub1/templates/tmpl1.html + sub2/templates/tmpl2.html + + Where sub1 and sub2 are prefixes, and both tmpl1.html and tmpl2.html + incldue foo.html. + """ + dir1 = os.path.join(self.dirname, 'templates') + os.mkdir(dir1) + file1 = open(os.path.join(dir1, 'foo.html'), 'w') + try: + file1.write("""<div>Included foo</div>""") + finally: + file1.close() + + dir2 = os.path.join(self.dirname, 'sub1', 'templates') + os.makedirs(dir2) + file2 = open(os.path.join(dir2, 'tmpl1.html'), 'w') + try: + file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="../foo.html" /> from sub1 + </html>""") + finally: + file2.close() + + dir3 = os.path.join(self.dirname, 'sub2', 'templates') + os.makedirs(dir3) + file3 = open(os.path.join(dir3, 'tmpl2.html'), 'w') + try: + file3.write("""<div>tmpl2</div>""") + finally: + file3.close() + + loader = TemplateLoader([dir1, TemplateLoader.prefixed( + sub1 = dir2, + sub2 = dir3 + )]) + tmpl = loader.load('sub1/tmpl1.html') + self.assertEqual("""<html> + <div>Included foo</div> from sub1 + </html>""", tmpl.generate().render()) + + def test_prefix_delegation_to_directories_with_subdirs(self): + """ + Test prefix delegation with the following layout: + + templates/foo.html + sub1/templates/tmpl1.html + sub1/templates/tmpl2.html + sub1/templates/bar/tmpl3.html + + Where sub1 is a prefix, and tmpl1.html includes all the others. + """ + dir1 = os.path.join(self.dirname, 'templates') + os.mkdir(dir1) + file1 = open(os.path.join(dir1, 'foo.html'), 'w') + try: + file1.write("""<div>Included foo</div>""") + finally: + file1.close() + + dir2 = os.path.join(self.dirname, 'sub1', 'templates') + os.makedirs(dir2) + file2 = open(os.path.join(dir2, 'tmpl1.html'), 'w') + try: + file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="../foo.html" /> from sub1 + <xi:include href="tmpl2.html" /> from sub1 + <xi:include href="bar/tmpl3.html" /> from sub1 + </html>""") + finally: + file2.close() + + file3 = open(os.path.join(dir2, 'tmpl2.html'), 'w') + try: + file3.write("""<div>tmpl2</div>""") + finally: + file3.close() + + dir3 = os.path.join(self.dirname, 'sub1', 'templates', 'bar') + os.makedirs(dir3) + file4 = open(os.path.join(dir3, 'tmpl3.html'), 'w') + try: + file4.write("""<div>bar/tmpl3</div>""") + finally: + file4.close() + + loader = TemplateLoader([dir1, TemplateLoader.prefixed( + sub1 = os.path.join(dir2), + sub2 = os.path.join(dir3) + )]) + tmpl = loader.load('sub1/tmpl1.html') + self.assertEqual("""<html> + <div>Included foo</div> from sub1 + <div>tmpl2</div> from sub1 + <div>bar/tmpl3</div> from sub1 + </html>""", tmpl.generate().render()) + def suite(): suite = unittest.TestSuite()
--- a/genshi/template/tests/markup.py +++ b/genshi/template/tests/markup.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -13,6 +13,7 @@ import doctest import os +import pickle import shutil from StringIO import StringIO import sys @@ -22,7 +23,7 @@ from genshi.core import Markup from genshi.input import XML from genshi.template.base import BadDirectiveError, TemplateSyntaxError -from genshi.template.loader import TemplateLoader +from genshi.template.loader import TemplateLoader, TemplateNotFound from genshi.template.markup import MarkupTemplate @@ -39,6 +40,15 @@ tmpl = MarkupTemplate(stream) self.assertEqual('<root> 42 42</root>', str(tmpl.generate(var=42))) + def test_pickle(self): + stream = XML('<root>$var</root>') + tmpl = MarkupTemplate(stream) + buf = StringIO() + pickle.dump(tmpl, buf, 2) + buf.seek(0) + unpickled = pickle.load(buf) + self.assertEqual('<root>42</root>', str(unpickled.generate(var=42))) + def test_interpolate_mixed3(self): tmpl = MarkupTemplate('<root> ${var} $var</root>') self.assertEqual('<root> 42 42</root>', str(tmpl.generate(var=42))) @@ -71,18 +81,16 @@ tmpl = MarkupTemplate(xml, filename='test.html') except BadDirectiveError, e: self.assertEqual('test.html', e.filename) - if sys.version_info[:2] >= (2, 4): - self.assertEqual(1, e.lineno) + self.assertEqual(1, e.lineno) def test_directive_value_syntax_error(self): xml = """<p xmlns:py="http://genshi.edgewall.org/" py:if="bar'" />""" try: - tmpl = MarkupTemplate(xml, filename='test.html') - self.fail('Expected SyntaxError') + tmpl = MarkupTemplate(xml, filename='test.html').generate() + self.fail('Expected TemplateSyntaxError') except TemplateSyntaxError, e: self.assertEqual('test.html', e.filename) - if sys.version_info[:2] >= (2, 4): - self.assertEqual(1, e.lineno) + self.assertEqual(1, e.lineno) def test_expression_syntax_error(self): xml = """<p> @@ -90,11 +98,10 @@ </p>""" try: tmpl = MarkupTemplate(xml, filename='test.html') - self.fail('Expected SyntaxError') + self.fail('Expected TemplateSyntaxError') except TemplateSyntaxError, e: self.assertEqual('test.html', e.filename) - if sys.version_info[:2] >= (2, 4): - self.assertEqual(2, e.lineno) + self.assertEqual(2, e.lineno) def test_expression_syntax_error_multi_line(self): xml = """<p><em></em> @@ -104,11 +111,10 @@ </p>""" try: tmpl = MarkupTemplate(xml, filename='test.html') - self.fail('Expected SyntaxError') + self.fail('Expected TemplateSyntaxError') except TemplateSyntaxError, e: self.assertEqual('test.html', e.filename) - if sys.version_info[:2] >= (2, 4): - self.assertEqual(3, e.lineno) + self.assertEqual(3, e.lineno) def test_markup_noescape(self): """ @@ -124,7 +130,8 @@ def test_text_noescape_quotes(self): """ - Verify that outputting context data in text nodes doesn't escape quotes. + Verify that outputting context data in text nodes doesn't escape + quotes. """ tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/"> $myvar @@ -195,6 +202,17 @@ \xf6 </div>""", unicode(tmpl.generate())) + def test_exec_with_trailing_space(self): + """ + Verify that a code block processing instruction with trailing space + does not cause a syntax error (see ticket #127). + """ + MarkupTemplate(u"""<foo> + <?python + bar = 42 + ?> + </foo>""") + def test_exec_import(self): tmpl = MarkupTemplate(u"""<?python from datetime import timedelta ?> <div xmlns:py="http://genshi.edgewall.org/"> @@ -259,7 +277,7 @@ finally: shutil.rmtree(dirname) - def test_dynamic_inlude_href(self): + def test_dynamic_include_href(self): dirname = tempfile.mkdtemp(suffix='genshi_test') try: file1 = open(os.path.join(dirname, 'tmpl1.html'), 'w') @@ -285,7 +303,7 @@ finally: shutil.rmtree(dirname) - def test_select_inluded_elements(self): + def test_select_included_elements(self): dirname = tempfile.mkdtemp(suffix='genshi_test') try: file1 = open(os.path.join(dirname, 'tmpl1.html'), 'w') @@ -340,6 +358,23 @@ finally: shutil.rmtree(dirname) + def test_error_when_include_not_found(self): + dirname = tempfile.mkdtemp(suffix='genshi_test') + try: + file2 = open(os.path.join(dirname, 'tmpl2.html'), 'w') + try: + file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="tmpl1.html"/> + </html>""") + finally: + file2.close() + + loader = TemplateLoader([dirname], auto_reload=True) + tmpl = loader.load('tmpl2.html') + self.assertRaises(TemplateNotFound, tmpl.generate().render) + finally: + shutil.rmtree(dirname) + def test_fallback_when_include_not_found(self): dirname = tempfile.mkdtemp(suffix='genshi_test') try: @@ -360,6 +395,26 @@ finally: shutil.rmtree(dirname) + def test_fallback_when_auto_reload_true(self): + dirname = tempfile.mkdtemp(suffix='genshi_test') + try: + file2 = open(os.path.join(dirname, 'tmpl2.html'), 'w') + try: + file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="tmpl1.html"><xi:fallback> + Missing</xi:fallback></xi:include> + </html>""") + finally: + file2.close() + + loader = TemplateLoader([dirname], auto_reload=True) + tmpl = loader.load('tmpl2.html') + self.assertEqual("""<html> + Missing + </html>""", tmpl.generate().render()) + finally: + shutil.rmtree(dirname) + def test_include_in_fallback(self): dirname = tempfile.mkdtemp(suffix='genshi_test') try: @@ -386,7 +441,7 @@ loader = TemplateLoader([dirname]) tmpl = loader.load('tmpl3.html') self.assertEqual("""<html> - <div>Included</div> + <div>Included</div> </html>""", tmpl.generate().render()) finally: shutil.rmtree(dirname) @@ -411,7 +466,36 @@ loader = TemplateLoader([dirname]) tmpl = loader.load('tmpl3.html') self.assertEqual("""<html> - Missing + Missing + </html>""", tmpl.generate().render()) + finally: + shutil.rmtree(dirname) + + def test_nested_include_in_fallback(self): + dirname = tempfile.mkdtemp(suffix='genshi_test') + try: + file1 = open(os.path.join(dirname, 'tmpl2.html'), 'w') + try: + file1.write("""<div>Included</div>""") + finally: + file1.close() + + file2 = open(os.path.join(dirname, 'tmpl3.html'), 'w') + try: + file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude"> + <xi:include href="tmpl2.html"> + <xi:fallback> + <xi:include href="tmpl1.html" /> + </xi:fallback> + </xi:include> + </html>""") + finally: + file2.close() + + loader = TemplateLoader([dirname]) + tmpl = loader.load('tmpl3.html') + self.assertEqual("""<html> + <div>Included</div> </html>""", tmpl.generate().render()) finally: shutil.rmtree(dirname) @@ -438,6 +522,212 @@ finally: shutil.rmtree(dirname) + def test_include_inlined(self): + dirname = tempfile.mkdtemp(suffix='genshi_test') + try: + file1 = open(os.path.join(dirname, 'tmpl1.html'), 'w') + try: + file1.write("""<div>Included</div>""") + finally: + file1.close() + + file2 = open(os.path.join(dirname, 'tmpl2.html'), 'w') + try: + file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude" + xmlns:py="http://genshi.edgewall.org/"> + <xi:include href="tmpl1.html" /> + </html>""") + finally: + file2.close() + + loader = TemplateLoader([dirname], auto_reload=False) + tmpl = loader.load('tmpl2.html') + # if not inlined the following would be 5 + self.assertEqual(7, len(tmpl.stream)) + self.assertEqual("""<html> + <div>Included</div> + </html>""", tmpl.generate().render()) + finally: + shutil.rmtree(dirname) + + def test_include_inlined_in_loop(self): + dirname = tempfile.mkdtemp(suffix='genshi_test') + try: + file1 = open(os.path.join(dirname, 'tmpl1.html'), 'w') + try: + file1.write("""<div>Included $idx</div>""") + finally: + file1.close() + + file2 = open(os.path.join(dirname, 'tmpl2.html'), 'w') + try: + file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude" + xmlns:py="http://genshi.edgewall.org/"> + <xi:include href="tmpl1.html" py:for="idx in range(3)" /> + </html>""") + finally: + file2.close() + + loader = TemplateLoader([dirname], auto_reload=False) + tmpl = loader.load('tmpl2.html') + self.assertEqual("""<html> + <div>Included 0</div><div>Included 1</div><div>Included 2</div> + </html>""", tmpl.generate().render()) + finally: + shutil.rmtree(dirname) + + def test_allow_exec_false(self): + xml = ("""<?python + title = "A Genshi Template" + ?> + <html xmlns:py="http://genshi.edgewall.org/"> + <head> + <title py:content="title">This is replaced.</title> + </head> + </html>""") + try: + tmpl = MarkupTemplate(xml, filename='test.html', + allow_exec=False) + self.fail('Expected SyntaxError') + except TemplateSyntaxError, e: + pass + + def test_allow_exec_true(self): + xml = ("""<?python + title = "A Genshi Template" + ?> + <html xmlns:py="http://genshi.edgewall.org/"> + <head> + <title py:content="title">This is replaced.</title> + </head> + </html>""") + tmpl = MarkupTemplate(xml, filename='test.html', allow_exec=True) + + def test_exec_in_match(self): + xml = ("""<html xmlns:py="http://genshi.edgewall.org/"> + <py:match path="body/p"> + <?python title="wakka wakka wakka" ?> + ${title} + </py:match> + <body><p>moot text</p></body> + </html>""") + tmpl = MarkupTemplate(xml, filename='test.html', allow_exec=True) + self.assertEqual("""<html> + <body> + wakka wakka wakka + </body> + </html>""", tmpl.generate().render()) + + def test_with_in_match(self): + xml = ("""<html xmlns:py="http://genshi.edgewall.org/"> + <py:match path="body/p"> + <h1>${select('text()')}</h1> + ${select('.')} + </py:match> + <body><p py:with="foo='bar'">${foo}</p></body> + </html>""") + tmpl = MarkupTemplate(xml, filename='test.html') + self.assertEqual("""<html> + <body> + <h1>bar</h1> + <p>bar</p> + </body> + </html>""", tmpl.generate().render()) + + def test_nested_include_matches(self): + # See ticket #157 + dirname = tempfile.mkdtemp(suffix='genshi_test') + try: + file1 = open(os.path.join(dirname, 'tmpl1.html'), 'w') + try: + file1.write("""<html xmlns:py="http://genshi.edgewall.org/" py:strip=""> + <div class="target">Some content.</div> +</html>""") + finally: + file1.close() + + file2 = open(os.path.join(dirname, 'tmpl2.html'), 'w') + try: + file2.write("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:xi="http://www.w3.org/2001/XInclude"> + <body> + <h1>Some full html document that includes file1.html</h1> + <xi:include href="tmpl1.html" /> + </body> +</html>""") + finally: + file2.close() + + file3 = open(os.path.join(dirname, 'tmpl3.html'), 'w') + try: + file3.write("""<html xmlns:py="http://genshi.edgewall.org/" + xmlns:xi="http://www.w3.org/2001/XInclude" py:strip=""> + <div py:match="div[@class='target']" py:attrs="select('@*')"> + Some added stuff. + ${select('*|text()')} + </div> + <xi:include href="tmpl2.html" /> +</html> +""") + finally: + file3.close() + + loader = TemplateLoader([dirname]) + tmpl = loader.load('tmpl3.html') + self.assertEqual(""" + <html> + <body> + <h1>Some full html document that includes file1.html</h1> + <div class="target"> + Some added stuff. + Some content. + </div> + </body> +</html> +""", tmpl.generate().render()) + finally: + shutil.rmtree(dirname) + + def test_nested_matches_without_buffering(self): + xml = ("""<html xmlns:py="http://genshi.edgewall.org/"> + <py:match path="body" once="true" buffer="false"> + <body> + ${select('*|text')} + And some other stuff... + </body> + </py:match> + <body> + <span py:match="span">Foo</span> + <span>Bar</span> + </body> + </html>""") + tmpl = MarkupTemplate(xml, filename='test.html') + self.assertEqual("""<html> + <body> + <span>Foo</span> + And some other stuff... + </body> + </html>""", tmpl.generate().render()) + + def test_match_without_select(self): + # See <http://genshi.edgewall.org/ticket/243> + xml = ("""<html xmlns:py="http://genshi.edgewall.org/"> + <py:match path="body" buffer="false"> + <body> + This replaces the other text. + </body> + </py:match> + <body> + This gets replaced. + </body> + </html>""") + tmpl = MarkupTemplate(xml, filename='test.html') + self.assertEqual("""<html> + <body> + This replaces the other text. + </body> + </html>""", tmpl.generate().render()) + def suite(): suite = unittest.TestSuite()
--- a/genshi/template/tests/plugin.py +++ b/genshi/template/tests/plugin.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006 Edgewall Software +# Copyright (C) 2006-2007 Edgewall Software # Copyright (C) 2006 Matthew Good # All rights reserved. # @@ -18,7 +18,7 @@ from genshi.core import Stream from genshi.output import DocType -from genshi.template import MarkupTemplate, TextTemplate +from genshi.template import MarkupTemplate, TextTemplate, NewTextTemplate from genshi.template.plugin import ConfigurationError, \ MarkupTemplateEnginePlugin, \ TextTemplateEnginePlugin @@ -145,6 +145,23 @@ </body> </html>""", output) + def test_render_fragment_with_doctype(self): + plugin = MarkupTemplateEnginePlugin(options={ + 'genshi.default_doctype': 'html-strict', + }) + tmpl = plugin.load_template(PACKAGE + '.templates.test_no_doctype') + output = plugin.render({'message': 'Hello'}, template=tmpl, + fragment=True) + self.assertEqual("""<html lang="en"> + <head> + <title>Test</title> + </head> + <body> + <h1>Test</h1> + <p>Hello</p> + </body> +</html>""", output) + def test_helper_functions(self): plugin = MarkupTemplateEnginePlugin() tmpl = plugin.load_template(PACKAGE + '.templates.functions') @@ -185,6 +202,15 @@ }) self.assertEqual('iso-8859-15', plugin.default_encoding) + def test_init_with_new_syntax(self): + plugin = TextTemplateEnginePlugin(options={ + 'genshi.new_text_syntax': 'yes', + }) + self.assertEqual(NewTextTemplate, plugin.template_class) + tmpl = plugin.load_template(PACKAGE + '.templates.new_syntax') + output = plugin.render({'foo': True}, template=tmpl) + self.assertEqual('bar', output) + def test_load_template_from_file(self): plugin = TextTemplateEnginePlugin() tmpl = plugin.load_template(PACKAGE + '.templates.test')
new file mode 100644 --- /dev/null +++ b/genshi/template/tests/templates/new_syntax.txt @@ -0,0 +1,1 @@ +{% if foo %}bar{% end %} \ No newline at end of file
new file mode 100644 --- /dev/null +++ b/genshi/template/tests/templates/test_no_doctype.html @@ -0,0 +1,13 @@ +<html xmlns="http://www.w3.org/1999/xhtml" + lang="en"> + + <head> + <title>Test</title> + </head> + + <body> + <h1>Test</h1> + <p>$message</p> + </body> + +</html>
--- a/genshi/template/tests/text.py +++ b/genshi/template/tests/text.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -17,11 +17,12 @@ import tempfile import unittest +from genshi.template.base import TemplateSyntaxError from genshi.template.loader import TemplateLoader -from genshi.template.text import TextTemplate +from genshi.template.text import OldTextTemplate, NewTextTemplate -class TextTemplateTestCase(unittest.TestCase): +class OldTextTemplateTestCase(unittest.TestCase): """Tests for text template processing.""" def setUp(self): @@ -31,19 +32,19 @@ shutil.rmtree(self.dirname) def test_escaping(self): - tmpl = TextTemplate('\\#escaped') + tmpl = OldTextTemplate('\\#escaped') self.assertEqual('#escaped', str(tmpl.generate())) def test_comment(self): - tmpl = TextTemplate('## a comment') + tmpl = OldTextTemplate('## a comment') self.assertEqual('', str(tmpl.generate())) def test_comment_escaping(self): - tmpl = TextTemplate('\\## escaped comment') + tmpl = OldTextTemplate('\\## escaped comment') self.assertEqual('## escaped comment', str(tmpl.generate())) def test_end_with_args(self): - tmpl = TextTemplate(""" + tmpl = OldTextTemplate(""" #if foo bar #end 'if foo'""") @@ -51,11 +52,16 @@ def test_latin1_encoded(self): text = u'$foo\xf6$bar'.encode('iso-8859-1') - tmpl = TextTemplate(text, encoding='iso-8859-1') + tmpl = OldTextTemplate(text, encoding='iso-8859-1') + self.assertEqual(u'x\xf6y', unicode(tmpl.generate(foo='x', bar='y'))) + + def test_unicode_input(self): + text = u'$foo\xf6$bar' + tmpl = OldTextTemplate(text) self.assertEqual(u'x\xf6y', unicode(tmpl.generate(foo='x', bar='y'))) def test_empty_lines1(self): - tmpl = TextTemplate("""Your items: + tmpl = OldTextTemplate("""Your items: #for item in items * ${item} @@ -65,10 +71,10 @@ * 0 * 1 * 2 -""", tmpl.generate(items=range(3)).render('text')) +""", tmpl.generate(items=range(3)).render()) def test_empty_lines2(self): - tmpl = TextTemplate("""Your items: + tmpl = OldTextTemplate("""Your items: #for item in items * ${item} @@ -82,7 +88,7 @@ * 2 -""", tmpl.generate(items=range(3)).render('text')) +""", tmpl.generate(items=range(3)).render()) def test_include(self): file1 = open(os.path.join(self.dirname, 'tmpl1.txt'), 'w') @@ -100,16 +106,159 @@ file2.close() loader = TemplateLoader([self.dirname]) - tmpl = loader.load('tmpl2.txt', cls=TextTemplate) + tmpl = loader.load('tmpl2.txt', cls=OldTextTemplate) self.assertEqual("""----- Included data below this line ----- Included ----- Included data above this line -----""", tmpl.generate().render()) - + + +class NewTextTemplateTestCase(unittest.TestCase): + """Tests for text template processing.""" + + def setUp(self): + self.dirname = tempfile.mkdtemp(suffix='markup_test') + + def tearDown(self): + shutil.rmtree(self.dirname) + + def test_escaping(self): + tmpl = NewTextTemplate('\\{% escaped %}') + self.assertEqual('{% escaped %}', str(tmpl.generate())) + + def test_comment(self): + tmpl = NewTextTemplate('{# a comment #}') + self.assertEqual('', str(tmpl.generate())) + + def test_comment_escaping(self): + tmpl = NewTextTemplate('\\{# escaped comment #}') + self.assertEqual('{# escaped comment #}', str(tmpl.generate())) + + def test_end_with_args(self): + tmpl = NewTextTemplate(""" +{% if foo %} + bar +{% end 'if foo' %}""") + self.assertEqual('\n', str(tmpl.generate(foo=False))) + + def test_latin1_encoded(self): + text = u'$foo\xf6$bar'.encode('iso-8859-1') + tmpl = NewTextTemplate(text, encoding='iso-8859-1') + self.assertEqual(u'x\xf6y', unicode(tmpl.generate(foo='x', bar='y'))) + + def test_unicode_input(self): + text = u'$foo\xf6$bar' + tmpl = NewTextTemplate(text) + self.assertEqual(u'x\xf6y', unicode(tmpl.generate(foo='x', bar='y'))) + + def test_empty_lines1(self): + tmpl = NewTextTemplate("""Your items: + +{% for item in items %}\ + * ${item} +{% end %}""") + self.assertEqual("""Your items: + + * 0 + * 1 + * 2 +""", tmpl.generate(items=range(3)).render()) + + def test_empty_lines2(self): + tmpl = NewTextTemplate("""Your items: + +{% for item in items %}\ + * ${item} + +{% end %}""") + self.assertEqual("""Your items: + + * 0 + + * 1 + + * 2 + +""", tmpl.generate(items=range(3)).render()) + + def test_exec_with_trailing_space(self): + """ + Verify that a code block with trailing space does not cause a syntax + error (see ticket #127). + """ + NewTextTemplate(u""" + {% python + bar = 42 + $} + """) + + def test_exec_import(self): + tmpl = NewTextTemplate(u"""{% python from datetime import timedelta %} + ${timedelta(days=2)} + """) + self.assertEqual(""" + 2 days, 0:00:00 + """, str(tmpl.generate())) + + def test_exec_def(self): + tmpl = NewTextTemplate(u"""{% python + def foo(): + return 42 + %} + ${foo()} + """) + self.assertEqual(u""" + 42 + """, str(tmpl.generate())) + + def test_include(self): + file1 = open(os.path.join(self.dirname, 'tmpl1.txt'), 'w') + try: + file1.write("Included") + finally: + file1.close() + + file2 = open(os.path.join(self.dirname, 'tmpl2.txt'), 'w') + try: + file2.write("""----- Included data below this line ----- +{% include tmpl1.txt %} +----- Included data above this line -----""") + finally: + file2.close() + + loader = TemplateLoader([self.dirname]) + tmpl = loader.load('tmpl2.txt', cls=NewTextTemplate) + self.assertEqual("""----- Included data below this line ----- +Included +----- Included data above this line -----""", tmpl.generate().render()) + + def test_include_expr(self): + file1 = open(os.path.join(self.dirname, 'tmpl1.txt'), 'w') + try: + file1.write("Included") + finally: + file1.close() + + file2 = open(os.path.join(self.dirname, 'tmpl2.txt'), 'w') + try: + file2.write("""----- Included data below this line ----- + {% include ${'%s.txt' % ('tmpl1',)} %} + ----- Included data above this line -----""") + finally: + file2.close() + + loader = TemplateLoader([self.dirname]) + tmpl = loader.load('tmpl2.txt', cls=NewTextTemplate) + self.assertEqual("""----- Included data below this line ----- + Included + ----- Included data above this line -----""", tmpl.generate().render()) + + def suite(): suite = unittest.TestSuite() - suite.addTest(doctest.DocTestSuite(TextTemplate.__module__)) - suite.addTest(unittest.makeSuite(TextTemplateTestCase, 'test')) + suite.addTest(doctest.DocTestSuite(NewTextTemplate.__module__)) + suite.addTest(unittest.makeSuite(OldTextTemplateTestCase, 'test')) + suite.addTest(unittest.makeSuite(NewTextTemplateTestCase, 'test')) return suite if __name__ == '__main__':
--- a/genshi/template/text.py +++ b/genshi/template/text.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -11,23 +11,235 @@ # individuals. For the exact contribution history, see the revision # history and logs, available at http://genshi.edgewall.org/log/. -"""Plain text templating engine.""" +"""Plain text templating engine. + +This module implements two template language syntaxes, at least for a certain +transitional period. `OldTextTemplate` (aliased to just `TextTemplate`) defines +a syntax that was inspired by Cheetah/Velocity. `NewTextTemplate` on the other +hand is inspired by the syntax of the Django template language, which has more +explicit delimiting of directives, and is more flexible with regards to +white space and line breaks. + +In a future release, `OldTextTemplate` will be phased out in favor of +`NewTextTemplate`, as the names imply. Therefore the new syntax is strongly +recommended for new projects, and existing projects may want to migrate to the +new syntax to remain compatible with future Genshi releases. +""" import re -from genshi.template.base import BadDirectiveError, Template, INCLUDE, SUB +from genshi.core import TEXT +from genshi.template.base import BadDirectiveError, Template, \ + TemplateSyntaxError, EXEC, INCLUDE, SUB +from genshi.template.eval import Suite from genshi.template.directives import * -from genshi.template.directives import Directive, _apply_directives +from genshi.template.directives import Directive from genshi.template.interpolation import interpolate -__all__ = ['TextTemplate'] +__all__ = ['NewTextTemplate', 'OldTextTemplate', 'TextTemplate'] __docformat__ = 'restructuredtext en' -class TextTemplate(Template): - """Implementation of a simple text-based template engine. +class NewTextTemplate(Template): + r"""Implementation of a simple text-based template engine. This class will + replace `OldTextTemplate` in a future release. - >>> tmpl = TextTemplate('''Dear $name, + It uses a more explicit delimiting style for directives: instead of the old + style which required putting directives on separate lines that were prefixed + with a ``#`` sign, directives and commenbtsr are enclosed in delimiter pairs + (by default ``{% ... %}`` and ``{# ... #}``, respectively). + + Variable substitution uses the same interpolation syntax as for markup + languages: simple references are prefixed with a dollar sign, more complex + expression enclosed in curly braces. + + >>> tmpl = NewTextTemplate('''Dear $name, + ... + ... {# This is a comment #} + ... We have the following items for you: + ... {% for item in items %} + ... * ${'Item %d' % item} + ... {% end %} + ... ''') + >>> print tmpl.generate(name='Joe', items=[1, 2, 3]).render() + Dear Joe, + <BLANKLINE> + <BLANKLINE> + We have the following items for you: + <BLANKLINE> + * Item 1 + <BLANKLINE> + * Item 2 + <BLANKLINE> + * Item 3 + <BLANKLINE> + <BLANKLINE> + + By default, no spaces or line breaks are removed. If a line break should + not be included in the output, prefix it with a backslash: + + >>> tmpl = NewTextTemplate('''Dear $name, + ... + ... {# This is a comment #}\ + ... We have the following items for you: + ... {% for item in items %}\ + ... * $item + ... {% end %}\ + ... ''') + >>> print tmpl.generate(name='Joe', items=[1, 2, 3]).render() + Dear Joe, + <BLANKLINE> + We have the following items for you: + * 1 + * 2 + * 3 + <BLANKLINE> + + Backslashes are also used to escape the start delimiter of directives and + comments: + + >>> tmpl = NewTextTemplate('''Dear $name, + ... + ... \{# This is a comment #} + ... We have the following items for you: + ... {% for item in items %}\ + ... * $item + ... {% end %}\ + ... ''') + >>> print tmpl.generate(name='Joe', items=[1, 2, 3]).render() + Dear Joe, + <BLANKLINE> + {# This is a comment #} + We have the following items for you: + * 1 + * 2 + * 3 + <BLANKLINE> + + :since: version 0.5 + """ + directives = [('def', DefDirective), + ('when', WhenDirective), + ('otherwise', OtherwiseDirective), + ('for', ForDirective), + ('if', IfDirective), + ('choose', ChooseDirective), + ('with', WithDirective)] + serializer = 'text' + + _DIRECTIVE_RE = r'((?<!\\)%s\s*(\w+)\s*(.*?)\s*%s|(?<!\\)%s.*?%s)' + _ESCAPE_RE = r'\\\n|\\(\\)|\\(%s)|\\(%s)' + + def __init__(self, source, filepath=None, filename=None, loader=None, + encoding=None, lookup='strict', allow_exec=False, + delims=('{%', '%}', '{#', '#}')): + self.delimiters = delims + Template.__init__(self, source, filepath=filepath, filename=filename, + loader=loader, encoding=encoding, lookup=lookup) + + def _get_delims(self): + return self._delims + def _set_delims(self, delims): + if len(delims) != 4: + raise ValueError('delimiers tuple must have exactly four elements') + self._delims = delims + self._directive_re = re.compile(self._DIRECTIVE_RE % tuple( + map(re.escape, delims) + ), re.DOTALL) + self._escape_re = re.compile(self._ESCAPE_RE % tuple( + map(re.escape, delims[::2]) + )) + delimiters = property(_get_delims, _set_delims, """\ + The delimiters for directives and comments. This should be a four item tuple + of the form ``(directive_start, directive_end, comment_start, + comment_end)``, where each item is a string. + """) + + def _parse(self, source, encoding): + """Parse the template from text input.""" + stream = [] # list of events of the "compiled" template + dirmap = {} # temporary mapping of directives to elements + depth = 0 + + source = source.read() + if isinstance(source, str): + source = source.decode(encoding or 'utf-8', 'replace') + offset = 0 + lineno = 1 + + _escape_sub = self._escape_re.sub + def _escape_repl(mo): + groups = filter(None, mo.groups()) + if not groups: + return '' + return groups[0] + + for idx, mo in enumerate(self._directive_re.finditer(source)): + start, end = mo.span(1) + if start > offset: + text = _escape_sub(_escape_repl, source[offset:start]) + for kind, data, pos in interpolate(text, self.filepath, lineno, + lookup=self.lookup): + stream.append((kind, data, pos)) + lineno += len(text.splitlines()) + + lineno += len(source[start:end].splitlines()) + command, value = mo.group(2, 3) + + if command == 'include': + pos = (self.filename, lineno, 0) + value = list(interpolate(value, self.filepath, lineno, 0, + lookup=self.lookup)) + if len(value) == 1 and value[0][0] is TEXT: + value = value[0][1] + stream.append((INCLUDE, (value, None, []), pos)) + + elif command == 'python': + if not self.allow_exec: + raise TemplateSyntaxError('Python code blocks not allowed', + self.filepath, lineno) + try: + suite = Suite(value, self.filepath, lineno, + lookup=self.lookup) + except SyntaxError, err: + raise TemplateSyntaxError(err, self.filepath, + lineno + (err.lineno or 1) - 1) + pos = (self.filename, lineno, 0) + stream.append((EXEC, suite, pos)) + + elif command == 'end': + depth -= 1 + if depth in dirmap: + directive, start_offset = dirmap.pop(depth) + substream = stream[start_offset:] + stream[start_offset:] = [(SUB, ([directive], substream), + (self.filepath, lineno, 0))] + + elif command: + cls = self.get_directive(command) + if cls is None: + raise BadDirectiveError(command) + directive = cls, value, None, (self.filepath, lineno, 0) + dirmap[depth] = (directive, len(stream)) + depth += 1 + + offset = end + + if offset < len(source): + text = _escape_sub(_escape_repl, source[offset:]) + for kind, data, pos in interpolate(text, self.filepath, lineno, + lookup=self.lookup): + stream.append((kind, data, pos)) + + return stream + + +class OldTextTemplate(Template): + """Legacy implementation of the old syntax text-based templates. This class + is provided in a transition phase for backwards compatibility. New code + should use the `NewTextTemplate` class and the improved syntax it provides. + + >>> tmpl = OldTextTemplate('''Dear $name, ... ... We have the following items for you: ... #for item in items @@ -36,7 +248,7 @@ ... ... All the best, ... Foobar''') - >>> print tmpl.generate(name='Joe', items=[1, 2, 3]).render('text') + >>> print tmpl.generate(name='Joe', items=[1, 2, 3]).render() Dear Joe, <BLANKLINE> We have the following items for you: @@ -54,6 +266,7 @@ ('if', IfDirective), ('choose', ChooseDirective), ('with', WithDirective)] + serializer = 'text' _DIRECTIVE_RE = re.compile(r'(?:^[ \t]*(?<!\\)#(end).*\n?)|' r'(?:^[ \t]*(?<!\\)#((?:\w+|#).*)\n?)', @@ -64,10 +277,10 @@ stream = [] # list of events of the "compiled" template dirmap = {} # temporary mapping of directives to elements depth = 0 - if not encoding: - encoding = 'utf-8' - source = source.read().decode(encoding, 'replace') + source = source.read() + if isinstance(source, str): + source = source.decode(encoding or 'utf-8', 'replace') offset = 0 lineno = 1 @@ -75,8 +288,7 @@ start, end = mo.span() if start > offset: text = source[offset:start] - for kind, data, pos in interpolate(text, self.basedir, - self.filename, lineno, + for kind, data, pos in interpolate(text, self.filepath, lineno, lookup=self.lookup): stream.append((kind, data, pos)) lineno += len(text.splitlines()) @@ -98,9 +310,9 @@ (self.filepath, lineno, 0))] elif command == 'include': pos = (self.filename, lineno, 0) - stream.append((INCLUDE, (value.strip(), []), pos)) + stream.append((INCLUDE, (value.strip(), None, []), pos)) elif command != '#': - cls = self._dir_by_name.get(command) + cls = self.get_directive(command) if cls is None: raise BadDirectiveError(command) directive = cls, value, None, (self.filepath, lineno, 0) @@ -111,9 +323,11 @@ if offset < len(source): text = source[offset:].replace('\\#', '#') - for kind, data, pos in interpolate(text, self.basedir, - self.filename, lineno, + for kind, data, pos in interpolate(text, self.filepath, lineno, lookup=self.lookup): stream.append((kind, data, pos)) return stream + + +TextTemplate = OldTextTemplate
--- a/genshi/tests/builder.py +++ b/genshi/tests/builder.py @@ -16,7 +16,7 @@ import unittest from genshi.builder import Element, tag -from genshi.core import Attrs, Stream +from genshi.core import Attrs, Markup, Stream from genshi.input import XML @@ -42,6 +42,15 @@ (None, -1, -1)), event) + def test_duplicate_attributes(self): + link = tag.a(href='#1', href_='#2')('Bar') + bits = iter(link.generate()) + self.assertEqual((Stream.START, + ('a', Attrs([('href', "#1")])), + (None, -1, -1)), bits.next()) + self.assertEqual((Stream.TEXT, u'Bar', (None, -1, -1)), bits.next()) + self.assertEqual((Stream.END, 'a', (None, -1, -1)), bits.next()) + def test_stream_as_child(self): xml = list(tag.span(XML('<b>Foo</b>')).generate()) self.assertEqual(5, len(xml)) @@ -51,6 +60,12 @@ self.assertEqual((Stream.END, 'b'), xml[3][:2]) self.assertEqual((Stream.END, 'span'), xml[4][:2]) + def test_markup_escape(self): + from genshi.core import Markup + m = Markup('See %s') % tag.a('genshi', + href='http://genshi.edgwall.org') + self.assertEqual(m, Markup('See <a href="http://genshi.edgwall.org">' + 'genshi</a>')) def suite(): suite = unittest.TestSuite()
--- a/genshi/tests/core.py +++ b/genshi/tests/core.py @@ -14,10 +14,14 @@ import doctest import pickle from StringIO import StringIO +try: + from cStringIO import StringIO as cStringIO +except ImportError: + cStringIO = StringIO import unittest from genshi import core -from genshi.core import Markup, Namespace, QName, escape, unescape +from genshi.core import Markup, Attrs, Namespace, QName, escape, unescape from genshi.input import XML, ParseError @@ -35,6 +39,18 @@ xml = XML('<li>Über uns</li>') self.assertEqual('<li>Über uns</li>', xml.render(encoding='ascii')) + def test_render_output_stream_utf8(self): + xml = XML('<li>Über uns</li>') + strio = cStringIO() + self.assertEqual(None, xml.render(out=strio)) + self.assertEqual('<li>Über uns</li>', strio.getvalue()) + + def test_render_output_stream_unicode(self): + xml = XML('<li>Über uns</li>') + strio = StringIO() + self.assertEqual(None, xml.render(encoding=None, out=strio)) + self.assertEqual(u'<li>Über uns</li>', strio.getvalue()) + def test_pickle(self): xml = XML('<li>Foo</li>') buf = StringIO() @@ -46,6 +62,10 @@ class MarkupTestCase(unittest.TestCase): + def test_new_with_encoding(self): + markup = Markup('Döner', encoding='utf-8') + self.assertEquals("<Markup u'D\\xf6ner'>", repr(markup)) + def test_repr(self): markup = Markup('foo') self.assertEquals("<Markup u'foo'>", repr(markup)) @@ -91,6 +111,16 @@ assert type(markup) is Markup self.assertEquals('<b>&</b> boo', markup) + def test_mod_mapping(self): + markup = Markup('<b>%(foo)s</b>') % {'foo': '&'} + assert type(markup) is Markup + self.assertEquals('<b>&</b>', markup) + + def test_mod_noescape(self): + markup = Markup('<b>%(amp)s</b>') % {'amp': Markup('&')} + assert type(markup) is Markup + self.assertEquals('<b>&</b>', markup) + def test_mul(self): markup = Markup('<b>foo</b>') * 2 assert type(markup) is Markup @@ -134,6 +164,18 @@ self.assertEquals("<Markup u'foo'>", repr(pickle.load(buf))) +class AttrsTestCase(unittest.TestCase): + + def test_pickle(self): + attrs = Attrs([("attr1", "foo"), ("attr2", "bar")]) + buf = StringIO() + pickle.dump(attrs, buf, 2) + buf.seek(0) + unpickled = pickle.load(buf) + self.assertEquals("Attrs([('attr1', 'foo'), ('attr2', 'bar')])", + repr(unpickled)) + + class NamespaceTestCase(unittest.TestCase): def test_pickle(self): @@ -165,12 +207,18 @@ self.assertEqual("QName(u'http://www.example.org/namespace}elem')", repr(QName('http://www.example.org/namespace}elem'))) + def test_leading_curly_brace(self): + qname = QName('{http://www.example.org/namespace}elem') + self.assertEquals('http://www.example.org/namespace', qname.namespace) + self.assertEquals('elem', qname.localname) + def suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(StreamTestCase, 'test')) suite.addTest(unittest.makeSuite(MarkupTestCase, 'test')) suite.addTest(unittest.makeSuite(NamespaceTestCase, 'test')) + suite.addTest(unittest.makeSuite(AttrsTestCase, 'test')) suite.addTest(unittest.makeSuite(QNameTestCase, 'test')) suite.addTest(doctest.DocTestSuite(core)) return suite
--- a/genshi/tests/input.py +++ b/genshi/tests/input.py @@ -28,8 +28,7 @@ kind, data, pos = events[1] self.assertEqual(Stream.TEXT, kind) self.assertEqual(u'foo bar', data) - if sys.version_info[:2] >= (2, 4): - self.assertEqual((None, 1, 6), pos) + self.assertEqual((None, 1, 6), pos) def test_text_node_pos_multi_line(self): text = '''<elem>foo @@ -38,8 +37,7 @@ kind, data, pos = events[1] self.assertEqual(Stream.TEXT, kind) self.assertEqual(u'foo\nbar', data) - if sys.version_info[:2] >= (2, 4): - self.assertEqual((None, 1, -1), pos) + self.assertEqual((None, 1, -1), pos) def test_element_attribute_order(self): text = '<elem title="baz" id="foo" class="bar" />' @@ -123,8 +121,7 @@ kind, data, pos = events[1] self.assertEqual(Stream.TEXT, kind) self.assertEqual(u'foo bar', data) - if sys.version_info[:2] >= (2, 4): - self.assertEqual((None, 1, 6), pos) + self.assertEqual((None, 1, 6), pos) def test_text_node_pos_multi_line(self): text = '''<elem>foo @@ -133,8 +130,7 @@ kind, data, pos = events[1] self.assertEqual(Stream.TEXT, kind) self.assertEqual(u'foo\nbar', data) - if sys.version_info[:2] >= (2, 4): - self.assertEqual((None, 1, 6), pos) + self.assertEqual((None, 1, 6), pos) def test_input_encoding_text(self): text = u'<div>\xf6</div>'.encode('iso-8859-1')
--- a/genshi/tests/output.py +++ b/genshi/tests/output.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- # -# Copyright (C) 2006-2007 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -23,6 +23,15 @@ class XMLSerializerTestCase(unittest.TestCase): + def test_with_xml_decl(self): + stream = Stream([(Stream.XML_DECL, ('1.0', None, -1), (None, -1, -1))]) + output = stream.render(XMLSerializer, doctype='xhtml') + self.assertEqual('<?xml version="1.0"?>\n' + '<!DOCTYPE html PUBLIC ' + '"-//W3C//DTD XHTML 1.0 Strict//EN" ' + '"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">\n', + output) + def test_doctype_in_stream(self): stream = Stream([(Stream.DOCTYPE, DocType.HTML_STRICT, (None, -1, -1))]) output = stream.render(XMLSerializer) @@ -194,6 +203,34 @@ class XHTMLSerializerTestCase(unittest.TestCase): + def test_xml_decl_dropped(self): + stream = Stream([(Stream.XML_DECL, ('1.0', None, -1), (None, -1, -1))]) + output = stream.render(XHTMLSerializer, doctype='xhtml') + self.assertEqual('<!DOCTYPE html PUBLIC ' + '"-//W3C//DTD XHTML 1.0 Strict//EN" ' + '"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">\n', + output) + + def test_xml_decl_included(self): + stream = Stream([(Stream.XML_DECL, ('1.0', None, -1), (None, -1, -1))]) + output = stream.render(XHTMLSerializer, doctype='xhtml', + drop_xml_decl=False) + self.assertEqual('<?xml version="1.0"?>\n' + '<!DOCTYPE html PUBLIC ' + '"-//W3C//DTD XHTML 1.0 Strict//EN" ' + '"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">\n', + output) + + def test_xml_lang(self): + text = '<p xml:lang="en">English text</p>' + output = XML(text).render(XHTMLSerializer) + self.assertEqual('<p lang="en" xml:lang="en">English text</p>', output) + + def test_xml_lang_nodup(self): + text = '<p xml:lang="en" lang="en">English text</p>' + output = XML(text).render(XHTMLSerializer) + self.assertEqual('<p xml:lang="en" lang="en">English text</p>', output) + def test_textarea_whitespace(self): content = '\nHey there. \n\n I am indented.\n' stream = XML('<textarea name="foo">%s</textarea>' % content) @@ -209,7 +246,7 @@ def test_xml_space(self): text = '<foo xml:space="preserve"> Do not mess \n\n with me </foo>' output = XML(text).render(XHTMLSerializer) - self.assertEqual(text, output) + self.assertEqual('<foo> Do not mess \n\n with me </foo>', output) def test_empty_script(self): text = """<html xmlns="http://www.w3.org/1999/xhtml"> @@ -324,6 +361,16 @@ class HTMLSerializerTestCase(unittest.TestCase): + def test_xml_lang(self): + text = '<p xml:lang="en">English text</p>' + output = XML(text).render(HTMLSerializer) + self.assertEqual('<p lang="en">English text</p>', output) + + def test_xml_lang_nodup(self): + text = '<p lang="en" xml:lang="en">English text</p>' + output = XML(text).render(HTMLSerializer) + self.assertEqual('<p lang="en">English text</p>', output) + def test_textarea_whitespace(self): content = '\nHey there. \n\n I am indented.\n' stream = XML('<textarea name="foo">%s</textarea>' % content) @@ -343,7 +390,7 @@ def test_empty_script(self): text = '<script src="foo.js" />' - output = XML(text).render(XHTMLSerializer) + output = XML(text).render(HTMLSerializer) self.assertEqual('<script src="foo.js"></script>', output) def test_script_escaping(self):
--- a/genshi/tests/path.py +++ b/genshi/tests/path.py @@ -15,11 +15,56 @@ import unittest from genshi.input import XML -from genshi.path import Path, PathSyntaxError +from genshi.path import Path, PathParser, PathSyntaxError, GenericStrategy, \ + SingleStepStrategy, SimplePathStrategy +class FakePath(Path): + def __init__(self, strategy): + self.strategy = strategy + def test(self, ignore_context = False): + return self.strategy.test(ignore_context) + class PathTestCase(unittest.TestCase): + strategies = [GenericStrategy, SingleStepStrategy, SimplePathStrategy] + def _create_path(self, expression, expected): + return path + + def _test_strategies(self, stream, path, render, + namespaces=None, variables=None): + for strategy in self.strategies: + if not strategy.supports(path): + continue + s = strategy(path) + rendered = FakePath(s).select(stream,namespaces=namespaces, + variables=variables).render() + msg = "Bad render using %s strategy"%str(strategy) + msg += "\nExpected:\t'%s'"%render + msg += "\nRendered:\t'%s'"%rendered + self.assertEqual(render, rendered, msg) + + def _test_expression(self, text, expected, stream=None, render="", + namespaces=None, variables=None): + path = Path(text) + if expected is not None: + self.assertEqual(expected, repr(path)) + + if stream is None: + return + + rendered = path.select(stream, namespaces=namespaces, + variables=variables).render() + msg = "Bad render using whole path" + msg += "\nExpected:\t'%s'"%render + msg += "\nRendered:\t'%s'"%rendered + self.assertEqual(render, rendered, msg) + + if len(path.paths) == 1: + self._test_strategies(stream, path.paths[0], render, + namespaces=namespaces, variables=variables) + + def test_error_no_absolute_path(self): self.assertRaises(PathSyntaxError, Path, '/root') @@ -30,413 +75,441 @@ def test_1step(self): xml = XML('<root><elem/></root>') - path = Path('elem') - self.assertEqual('<Path "child::elem">', repr(path)) - self.assertEqual('<elem/>', path.select(xml).render()) - - path = Path('child::elem') - self.assertEqual('<Path "child::elem">', repr(path)) - self.assertEqual('<elem/>', path.select(xml).render()) + self._test_expression( 'elem', + '<Path "child::elem">', + xml, + '<elem/>') - path = Path('//elem') - self.assertEqual('<Path "descendant-or-self::node()/child::elem">', - repr(path)) - self.assertEqual('<elem/>', path.select(xml).render()) + self._test_expression( 'elem', + '<Path "child::elem">', + xml, + '<elem/>') - path = Path('descendant::elem') - self.assertEqual('<Path "descendant::elem">', repr(path)) - self.assertEqual('<elem/>', path.select(xml).render()) + self._test_expression( 'child::elem', + '<Path "child::elem">', + xml, + '<elem/>') + + self._test_expression( '//elem', + '<Path "descendant-or-self::elem">', + xml, + '<elem/>') + + self._test_expression( 'descendant::elem', + '<Path "descendant::elem">', + xml, + '<elem/>') def test_1step_self(self): xml = XML('<root><elem/></root>') - path = Path('.') - self.assertEqual('<Path "self::node()">', repr(path)) - self.assertEqual('<root><elem/></root>', path.select(xml).render()) + self._test_expression( '.', + '<Path "self::node()">', + xml, + '<root><elem/></root>') - path = Path('self::node()') - self.assertEqual('<Path "self::node()">', repr(path)) - self.assertEqual('<root><elem/></root>', path.select(xml).render()) + self._test_expression( 'self::node()', + '<Path "self::node()">', + xml, + '<root><elem/></root>') def test_1step_wildcard(self): xml = XML('<root><elem/></root>') - path = Path('*') - self.assertEqual('<Path "child::*">', repr(path)) - self.assertEqual('<elem/>', path.select(xml).render()) - - path = Path('child::*') - self.assertEqual('<Path "child::*">', repr(path)) - self.assertEqual('<elem/>', path.select(xml).render()) + self._test_expression( '*', + '<Path "child::*">', + xml, + '<elem/>') - path = Path('child::node()') - self.assertEqual('<Path "child::node()">', repr(path)) - self.assertEqual('<elem/>', Path('child::node()').select(xml).render()) + self._test_expression( 'child::*', + '<Path "child::*">', + xml, + '<elem/>') - path = Path('//*') - self.assertEqual('<Path "descendant-or-self::node()/child::*">', - repr(path)) - self.assertEqual('<elem/>', path.select(xml).render()) + self._test_expression( 'child::node()', + '<Path "child::node()">', + xml, + '<elem/>') + + self._test_expression( '//*', + '<Path "descendant-or-self::*">', + xml, + '<root><elem/></root>') def test_1step_attribute(self): - path = Path('@foo') - self.assertEqual('<Path "attribute::foo">', repr(path)) - - xml = XML('<root/>') - self.assertEqual('', path.select(xml).render()) + self._test_expression( '@foo', + '<Path "attribute::foo">', + XML('<root/>'), + '') xml = XML('<root foo="bar"/>') - self.assertEqual('bar', path.select(xml).render()) - path = Path('./@foo') - self.assertEqual('<Path "self::node()/attribute::foo">', repr(path)) - self.assertEqual('bar', path.select(xml).render()) + self._test_expression( '@foo', + '<Path "attribute::foo">', + xml, + 'bar') + + self._test_expression( './@foo', + '<Path "self::node()/attribute::foo">', + xml, + 'bar') def test_1step_text(self): xml = XML('<root>Hey</root>') - path = Path('text()') - self.assertEqual('<Path "child::text()">', repr(path)) - self.assertEqual('Hey', path.select(xml).render()) - - path = Path('./text()') - self.assertEqual('<Path "self::node()/child::text()">', repr(path)) - self.assertEqual('Hey', path.select(xml).render()) + self._test_expression( 'text()', + '<Path "child::text()">', + xml, + 'Hey') - path = Path('//text()') - self.assertEqual('<Path "descendant-or-self::node()/child::text()">', - repr(path)) - self.assertEqual('Hey', path.select(xml).render()) + self._test_expression( './text()', + '<Path "self::node()/child::text()">', + xml, + 'Hey') - path = Path('.//text()') - self.assertEqual('<Path "self::node()/descendant-or-self::node()/child::text()">', - repr(path)) - self.assertEqual('Hey', path.select(xml).render()) + self._test_expression( '//text()', + '<Path "descendant-or-self::text()">', + xml, + 'Hey') + + self._test_expression( './/text()', + '<Path "self::node()/descendant-or-self::node()/child::text()">', + xml, + 'Hey') def test_2step(self): xml = XML('<root><foo/><bar/></root>') - self.assertEqual('<foo/><bar/>', Path('*').select(xml).render()) - self.assertEqual('<bar/>', Path('bar').select(xml).render()) - self.assertEqual('', Path('baz').select(xml).render()) + self._test_expression('*', None, xml, '<foo/><bar/>') + self._test_expression('bar', None, xml, '<bar/>') + self._test_expression('baz', None, xml, '') def test_2step_attribute(self): xml = XML('<elem class="x"><span id="joe">Hey Joe</span></elem>') - self.assertEqual('x', Path('@*').select(xml).render()) - self.assertEqual('x', Path('./@*').select(xml).render()) - self.assertEqual('xjoe', Path('.//@*').select(xml).render()) - self.assertEqual('joe', Path('*/@*').select(xml).render()) + self._test_expression('@*', None, xml, 'x') + self._test_expression('./@*', None, xml, 'x') + self._test_expression('.//@*', None, xml, 'xjoe') + self._test_expression('*/@*', None, xml, 'joe') xml = XML('<elem><foo id="1"/><foo id="2"/></elem>') - self.assertEqual('', Path('@*').select(xml).render()) - self.assertEqual('12', Path('foo/@*').select(xml).render()) + self._test_expression('@*', None, xml, '') + self._test_expression('foo/@*', None, xml, '12') def test_2step_complex(self): xml = XML('<root><foo><bar/></foo></root>') - path = Path('foo/bar') - self.assertEqual('<Path "child::foo/child::bar">', repr(path)) - self.assertEqual('<bar/>', path.select(xml).render()) + self._test_expression( 'foo/bar', + '<Path "child::foo/child::bar">', + xml, + '<bar/>') - path = Path('./bar') - self.assertEqual('<Path "self::node()/child::bar">', repr(path)) - self.assertEqual('', path.select(xml).render()) + self._test_expression( './bar', + '<Path "self::node()/child::bar">', + xml, + '') - path = Path('foo/*') - self.assertEqual('<Path "child::foo/child::*">', repr(path)) - self.assertEqual('<bar/>', path.select(xml).render()) + self._test_expression( 'foo/*', + '<Path "child::foo/child::*">', + xml, + '<bar/>') xml = XML('<root><foo><bar id="1"/></foo><bar id="2"/></root>') - path = Path('./bar') - self.assertEqual('<Path "self::node()/child::bar">', repr(path)) - self.assertEqual('<bar id="2"/>', path.select(xml).render()) + self._test_expression( './bar', + '<Path "self::node()/child::bar">', + xml, + '<bar id="2"/>') def test_2step_text(self): xml = XML('<root><item>Foo</item></root>') - path = Path('item/text()') - self.assertEqual('<Path "child::item/child::text()">', repr(path)) - self.assertEqual('Foo', path.select(xml).render()) - - path = Path('*/text()') - self.assertEqual('<Path "child::*/child::text()">', repr(path)) - self.assertEqual('Foo', path.select(xml).render()) + self._test_expression( 'item/text()', + '<Path "child::item/child::text()">', + xml, + 'Foo') - path = Path('//text()') - self.assertEqual('<Path "descendant-or-self::node()/child::text()">', - repr(path)) - self.assertEqual('Foo', path.select(xml).render()) + self._test_expression( '*/text()', + '<Path "child::*/child::text()">', + xml, + 'Foo') - path = Path('./text()') - self.assertEqual('<Path "self::node()/child::text()">', repr(path)) - self.assertEqual('', path.select(xml).render()) + self._test_expression( '//text()', + '<Path "descendant-or-self::text()">', + xml, + 'Foo') + + self._test_expression( './text()', + '<Path "self::node()/child::text()">', + xml, + '') xml = XML('<root><item>Foo</item><item>Bar</item></root>') - path = Path('item/text()') - self.assertEqual('<Path "child::item/child::text()">', repr(path)) - self.assertEqual('FooBar', path.select(xml).render()) - - xml = XML('<root><item>Foo</item><item>Bar</item></root>') - self.assertEqual('FooBar', path.select(xml).render()) + self._test_expression( 'item/text()', + '<Path "child::item/child::text()">', + xml, + 'FooBar') def test_3step(self): xml = XML('<root><foo><bar/></foo></root>') - path = Path('foo/*') - self.assertEqual('<Path "child::foo/child::*">', repr(path)) - self.assertEqual('<bar/>', path.select(xml).render()) + self._test_expression( 'foo/*', + '<Path "child::foo/child::*">', + xml, + '<bar/>') def test_3step_complex(self): xml = XML('<root><foo><bar/></foo></root>') - path = Path('*/bar') - self.assertEqual('<Path "child::*/child::bar">', repr(path)) - self.assertEqual('<bar/>', path.select(xml).render()) + self._test_expression( '*/bar', + '<Path "child::*/child::bar">', + xml, + '<bar/>') xml = XML('<root><foo><bar id="1"/></foo><bar id="2"/></root>') - path = Path('//bar') - self.assertEqual('<Path "descendant-or-self::node()/child::bar">', - repr(path)) - self.assertEqual('<bar id="1"/><bar id="2"/>', - path.select(xml).render()) + self._test_expression( '//bar', + '<Path "descendant-or-self::bar">', + xml, + '<bar id="1"/><bar id="2"/>') def test_node_type_comment(self): xml = XML('<root><!-- commented --></root>') - path = Path('comment()') - self.assertEqual('<Path "child::comment()">', repr(path)) - self.assertEqual('<!-- commented -->', path.select(xml).render()) + self._test_expression( 'comment()', + '<Path "child::comment()">', + xml, + '<!-- commented -->') def test_node_type_text(self): xml = XML('<root>Some text <br/>in here.</root>') - path = Path('text()') - self.assertEqual('<Path "child::text()">', repr(path)) - self.assertEqual('Some text in here.', path.select(xml).render()) + self._test_expression( 'text()', + '<Path "child::text()">', + xml, + 'Some text in here.') def test_node_type_node(self): xml = XML('<root>Some text <br/>in here.</root>') - path = Path('node()') - self.assertEqual('<Path "child::node()">', repr(path)) - self.assertEqual('Some text <br/>in here.', path.select(xml).render()) + self._test_expression( 'node()', + '<Path "child::node()">', + xml, + 'Some text <br/>in here.',) def test_node_type_processing_instruction(self): xml = XML('<?python x = 2 * 3 ?><root><?php echo("x") ?></root>') - path = Path('processing-instruction()') - self.assertEqual('<Path "child::processing-instruction()">', - repr(path)) - self.assertEqual('<?python x = 2 * 3 ?><?php echo("x") ?>', - path.select(xml).render()) + self._test_expression( '//processing-instruction()', + '<Path "descendant-or-self::processing-instruction()">', + xml, + '<?python x = 2 * 3 ?><?php echo("x") ?>') - path = Path('processing-instruction("php")') - self.assertEqual('<Path "child::processing-instruction(\"php\")">', - repr(path)) - self.assertEqual('<?php echo("x") ?>', path.select(xml).render()) + self._test_expression( 'processing-instruction()', + '<Path "child::processing-instruction()">', + xml, + '<?php echo("x") ?>') + + self._test_expression( 'processing-instruction("php")', + '<Path "child::processing-instruction(\"php\")">', + xml, + '<?php echo("x") ?>') def test_simple_union(self): xml = XML("""<body>1<br />2<br />3<br /></body>""") - path = Path('*|text()') - self.assertEqual('<Path "child::*|child::text()">', repr(path)) - self.assertEqual('1<br/>2<br/>3<br/>', path.select(xml).render()) + self._test_expression( '*|text()', + '<Path "child::*|child::text()">', + xml, + '1<br/>2<br/>3<br/>') def test_predicate_name(self): xml = XML('<root><foo/><bar/></root>') - path = Path('*[name()="foo"]') - self.assertEqual('<foo/>', path.select(xml).render()) + self._test_expression('*[name()="foo"]', None, xml, '<foo/>') def test_predicate_localname(self): xml = XML('<root><foo xmlns="NS"/><bar/></root>') - path = Path('*[local-name()="foo"]') - self.assertEqual('<foo xmlns="NS"/>', path.select(xml).render()) + self._test_expression('*[local-name()="foo"]', None, xml, + '<foo xmlns="NS"/>') def test_predicate_namespace(self): xml = XML('<root><foo xmlns="NS"/><bar/></root>') - path = Path('*[namespace-uri()="NS"]') - self.assertEqual('<foo xmlns="NS"/>', path.select(xml).render()) + self._test_expression('*[namespace-uri()="NS"]', None, xml, + '<foo xmlns="NS"/>') def test_predicate_not_name(self): xml = XML('<root><foo/><bar/></root>') - path = Path('*[not(name()="foo")]') - self.assertEqual('<bar/>', path.select(xml).render()) + self._test_expression('*[not(name()="foo")]', None, xml, '<bar/>') def test_predicate_attr(self): xml = XML('<root><item/><item important="very"/></root>') - path = Path('item[@important]') - self.assertEqual('<item important="very"/>', path.select(xml).render()) - path = Path('item[@important="very"]') - self.assertEqual('<item important="very"/>', path.select(xml).render()) + self._test_expression('item[@important]', None, xml, + '<item important="very"/>') + self._test_expression('item[@important="very"]', None, xml, + '<item important="very"/>') def test_predicate_attr_equality(self): xml = XML('<root><item/><item important="notso"/></root>') - path = Path('item[@important="very"]') - self.assertEqual('', path.select(xml).render()) - path = Path('item[@important!="very"]') - self.assertEqual('<item/><item important="notso"/>', - path.select(xml).render()) + self._test_expression('item[@important="very"]', None, xml, '') + self._test_expression('item[@important!="very"]', None, xml, + '<item/><item important="notso"/>') def test_predicate_attr_greater_than(self): xml = XML('<root><item priority="3"/></root>') - path = Path('item[@priority>3]') - self.assertEqual('', path.select(xml).render()) - path = Path('item[@priority>2]') - self.assertEqual('<item priority="3"/>', path.select(xml).render()) + self._test_expression('item[@priority>3]', None, xml, '') + self._test_expression('item[@priority>2]', None, xml, + '<item priority="3"/>') def test_predicate_attr_less_than(self): xml = XML('<root><item priority="3"/></root>') - path = Path('item[@priority<3]') - self.assertEqual('', path.select(xml).render()) - path = Path('item[@priority<4]') - self.assertEqual('<item priority="3"/>', path.select(xml).render()) + self._test_expression('item[@priority<3]', None, xml, '') + self._test_expression('item[@priority<4]', None, xml, + '<item priority="3"/>') def test_predicate_attr_and(self): xml = XML('<root><item/><item important="very"/></root>') - path = Path('item[@important and @important="very"]') - self.assertEqual('<item important="very"/>', path.select(xml).render()) - path = Path('item[@important and @important="notso"]') - self.assertEqual('', path.select(xml).render()) + self._test_expression('item[@important and @important="very"]', + None, xml, '<item important="very"/>') + self._test_expression('item[@important and @important="notso"]', + None, xml, '') def test_predicate_attr_or(self): xml = XML('<root><item/><item important="very"/></root>') - path = Path('item[@urgent or @important]') - self.assertEqual('<item important="very"/>', path.select(xml).render()) - path = Path('item[@urgent or @notso]') - self.assertEqual('', path.select(xml).render()) + self._test_expression('item[@urgent or @important]', None, xml, + '<item important="very"/>') + self._test_expression('item[@urgent or @notso]', None, xml, '') def test_predicate_boolean_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[boolean("")]') - self.assertEqual('', path.select(xml).render()) - path = Path('*[boolean("yo")]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) - path = Path('*[boolean(0)]') - self.assertEqual('', path.select(xml).render()) - path = Path('*[boolean(42)]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) - path = Path('*[boolean(false())]') - self.assertEqual('', path.select(xml).render()) - path = Path('*[boolean(true())]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[boolean("")]', None, xml, '') + self._test_expression('*[boolean("yo")]', None, xml, '<foo>bar</foo>') + self._test_expression('*[boolean(0)]', None, xml, '') + self._test_expression('*[boolean(42)]', None, xml, '<foo>bar</foo>') + self._test_expression('*[boolean(false())]', None, xml, '') + self._test_expression('*[boolean(true())]', None, xml, + '<foo>bar</foo>') def test_predicate_ceil_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[ceiling("4.5")=5]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[ceiling("4.5")=5]', None, xml, + '<foo>bar</foo>') def test_predicate_concat_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[name()=concat("f", "oo")]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[name()=concat("f", "oo")]', None, xml, + '<foo>bar</foo>') def test_predicate_contains_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[contains(name(), "oo")]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[contains(name(), "oo")]', None, xml, + '<foo>bar</foo>') + + def test_predicate_matches_function(self): + xml = XML('<root><foo>bar</foo><bar>foo</bar></root>') + self._test_expression('*[matches(name(), "foo|bar")]', None, xml, + '<foo>bar</foo><bar>foo</bar>') def test_predicate_false_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[false()]') - self.assertEqual('', path.select(xml).render()) + self._test_expression('*[false()]', None, xml, '') def test_predicate_floor_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[floor("4.5")=4]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[floor("4.5")=4]', None, xml, + '<foo>bar</foo>') def test_predicate_normalize_space_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[normalize-space(" foo bar ")="foo bar"]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[normalize-space(" foo bar ")="foo bar"]', + None, xml, '<foo>bar</foo>') def test_predicate_number_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[number("3.0")=3]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) - path = Path('*[number("3.0")=3.0]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) - path = Path('*[number("0.1")=.1]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[number("3.0")=3]', None, xml, + '<foo>bar</foo>') + self._test_expression('*[number("3.0")=3.0]', None, xml, + '<foo>bar</foo>') + self._test_expression('*[number("0.1")=.1]', None, xml, + '<foo>bar</foo>') def test_predicate_round_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[round("4.4")=4]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) - path = Path('*[round("4.6")=5]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[round("4.4")=4]', None, xml, + '<foo>bar</foo>') + self._test_expression('*[round("4.6")=5]', None, xml, + '<foo>bar</foo>') def test_predicate_starts_with_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[starts-with(name(), "f")]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) - path = Path('*[starts-with(name(), "b")]') - self.assertEqual('', path.select(xml).render()) + self._test_expression('*[starts-with(name(), "f")]', None, xml, + '<foo>bar</foo>') + self._test_expression('*[starts-with(name(), "b")]', None, xml, '') def test_predicate_string_length_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[string-length(name())=3]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[string-length(name())=3]', None, xml, + '<foo>bar</foo>') def test_predicate_substring_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[substring(name(), 1)="oo"]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) - path = Path('*[substring(name(), 1, 1)="o"]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[substring(name(), 1)="oo"]', None, xml, + '<foo>bar</foo>') + self._test_expression('*[substring(name(), 1, 1)="o"]', None, xml, + '<foo>bar</foo>') def test_predicate_substring_after_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[substring-after(name(), "f")="oo"]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[substring-after(name(), "f")="oo"]', None, xml, + '<foo>bar</foo>') def test_predicate_substring_before_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[substring-before(name(), "oo")="f"]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[substring-before(name(), "oo")="f"]', + None, xml, '<foo>bar</foo>') def test_predicate_translate_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[translate(name(), "fo", "ba")="baa"]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[translate(name(), "fo", "ba")="baa"]', + None, xml, '<foo>bar</foo>') def test_predicate_true_function(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[true()]') - self.assertEqual('<foo>bar</foo>', path.select(xml).render()) + self._test_expression('*[true()]', None, xml, '<foo>bar</foo>') def test_predicate_variable(self): xml = XML('<root><foo>bar</foo></root>') - path = Path('*[name()=$bar]') variables = {'bar': 'foo'} - self.assertEqual('<foo>bar</foo>', - path.select(xml, variables=variables).render()) + self._test_expression('*[name()=$bar]', None, xml, '<foo>bar</foo>', + variables = variables) def test_predicate_position(self): xml = XML('<root><foo id="a1"/><foo id="a2"/><foo id="a3"/></root>') - path = Path('*[2]') - self.assertEqual('<foo id="a2"/>', path.select(xml).render()) + self._test_expression('*[2]', None, xml, '<foo id="a2"/>') def test_predicate_attr_and_position(self): xml = XML('<root><foo/><foo id="a1"/><foo id="a2"/></root>') - path = Path('*[@id][2]') - self.assertEqual('<foo id="a2"/>', path.select(xml).render()) + self._test_expression('*[@id][2]', None, xml, '<foo id="a2"/>') def test_predicate_position_and_attr(self): xml = XML('<root><foo/><foo id="a1"/><foo id="a2"/></root>') - path = Path('*[1][@id]') - self.assertEqual('', path.select(xml).render()) - path = Path('*[2][@id]') - self.assertEqual('<foo id="a1"/>', path.select(xml).render()) + self._test_expression('*[1][@id]', None, xml, '') + self._test_expression('*[2][@id]', None, xml, '<foo id="a1"/>') + + def test_predicate_advanced_position(self): + xml = XML('<root><a><b><c><d><e/></d></c></b></a></root>') + self._test_expression( 'descendant-or-self::*/' + 'descendant-or-self::*/' + 'descendant-or-self::*[2]/' + 'self::*/descendant::*[3]', None, xml, + '<d><e/></d>') + + def test_predicate_child_position(self): + xml = XML('\ +<root><a><b>1</b><b>2</b><b>3</b></a><a><b>4</b><b>5</b></a></root>') + self._test_expression('//a/b[2]', None, xml, '<b>2</b><b>5</b>') + self._test_expression('//a/b[3]', None, xml, '<b>3</b>') def test_name_with_namespace(self): xml = XML('<root xmlns:f="FOO"><f:foo>bar</f:foo></root>') - path = Path('f:foo') - self.assertEqual('<Path "child::f:foo">', repr(path)) - namespaces = {'f': 'FOO'} - self.assertEqual('<foo xmlns="FOO">bar</foo>', - path.select(xml, namespaces=namespaces).render()) + self._test_expression('f:foo', '<Path "child::f:foo">', xml, + '<foo xmlns="FOO">bar</foo>', + namespaces = {'f': 'FOO'}) def test_wildcard_with_namespace(self): xml = XML('<root xmlns:f="FOO"><f:foo>bar</f:foo></root>') - path = Path('f:*') - self.assertEqual('<Path "child::f:*">', repr(path)) - namespaces = {'f': 'FOO'} - self.assertEqual('<foo xmlns="FOO">bar</foo>', - path.select(xml, namespaces=namespaces).render()) + self._test_expression('f:*', '<Path "child::f:*">', xml, + '<foo xmlns="FOO">bar</foo>', + namespaces = {'f': 'FOO'}) def test_predicate_termination(self): """ @@ -444,25 +517,70 @@ cause an infinite loop. See <http://genshi.edgewall.org/ticket/82>. """ xml = XML('<ul flag="1"><li>a</li><li>b</li></ul>') - path = Path('.[@flag="1"]/*') - self.assertEqual('<li>a</li><li>b</li>', path.select(xml).render()) + self._test_expression('.[@flag="1"]/*', None, xml, + '<li>a</li><li>b</li>') xml = XML('<ul flag="1"><li>a</li><li>b</li></ul>') - path = Path('.[@flag="0"]/*') - self.assertEqual('', path.select(xml).render()) + self._test_expression('.[@flag="0"]/*', None, xml, '') def test_attrname_with_namespace(self): xml = XML('<root xmlns:f="FOO"><foo f:bar="baz"/></root>') - path = Path('foo[@f:bar]') - self.assertEqual('<foo xmlns:ns1="FOO" ns1:bar="baz"/>', - path.select(xml, namespaces={'f': 'FOO'}).render()) + self._test_expression('foo[@f:bar]', None, xml, + '<foo xmlns:ns1="FOO" ns1:bar="baz"/>', + namespaces={'f': 'FOO'}) def test_attrwildcard_with_namespace(self): xml = XML('<root xmlns:f="FOO"><foo f:bar="baz"/></root>') - path = Path('foo[@f:*]') - self.assertEqual('<foo xmlns:ns1="FOO" ns1:bar="baz"/>', - path.select(xml, namespaces={'f': 'FOO'}).render()) + self._test_expression('foo[@f:*]', None, xml, + '<foo xmlns:ns1="FOO" ns1:bar="baz"/>', + namespaces={'f': 'FOO'}) + def test_self_and_descendant(self): + xml = XML('<root><foo/></root>') + self._test_expression('self::root', None, xml, '<root><foo/></root>') + self._test_expression('self::foo', None, xml, '') + self._test_expression('descendant::root', None, xml, '') + self._test_expression('descendant::foo', None, xml, '<foo/>') + self._test_expression('descendant-or-self::root', None, xml, + '<root><foo/></root>') + self._test_expression('descendant-or-self::foo', None, xml, '<foo/>') + def test_long_simple_paths(self): + xml = XML('<root><a><b><a><d><a><b><a><b><a><b><a><c>!' + '</c></a></b></a></b></a></b></a></d></a></b></a></root>') + self._test_expression('//a/b/a/b/a/c', None, xml, '<c>!</c>') + self._test_expression('//a/b/a/c', None, xml, '<c>!</c>') + self._test_expression('//a/c', None, xml, '<c>!</c>') + self._test_expression('//c', None, xml, '<c>!</c>') + # Please note that a//b is NOT the same as a/descendant::b + # it is a/descendant-or-self::node()/b, which SimplePathStrategy + # does NOT support + self._test_expression('a/b/descendant::a/c', None, xml, '<c>!</c>') + self._test_expression('a/b/descendant::a/d/descendant::a/c', + None, xml, '<c>!</c>') + self._test_expression('a/b/descendant::a/d/a/c', None, xml, '') + self._test_expression('//d/descendant::b/descendant::b/descendant::b' + '/descendant::c', None, xml, '<c>!</c>') + self._test_expression('//d/descendant::b/descendant::b/descendant::b' + '/descendant::b/descendant::c', None, xml, '') + def _test_support(self, strategy_class, text): + path = PathParser(text, None, -1).parse()[0] + return strategy_class.supports(path) + def test_simple_strategy_support(self): + self.assert_(self._test_support(SimplePathStrategy, 'a/b')) + self.assert_(self._test_support(SimplePathStrategy, 'self::a/b')) + self.assert_(self._test_support(SimplePathStrategy, 'descendant::a/b')) + self.assert_(self._test_support(SimplePathStrategy, + 'descendant-or-self::a/b')) + self.assert_(self._test_support(SimplePathStrategy, '//a/b')) + self.assert_(self._test_support(SimplePathStrategy, 'a/@b')) + self.assert_(self._test_support(SimplePathStrategy, 'a/text()')) + + # a//b is a/descendant-or-self::node()/b + self.assert_(not self._test_support(SimplePathStrategy, 'a//b')) + self.assert_(not self._test_support(SimplePathStrategy, 'node()/@a')) + self.assert_(not self._test_support(SimplePathStrategy, '@a')) + self.assert_(not self._test_support(SimplePathStrategy, 'foo:bar')) + self.assert_(not self._test_support(SimplePathStrategy, 'a/@foo:bar')) def suite(): suite = unittest.TestSuite()
--- a/genshi/util.py +++ b/genshi/util.py @@ -152,7 +152,7 @@ """ retval = [] for item in items: - if isinstance(item, (list, tuple)): + if isinstance(item, (frozenset, list, set, tuple)): retval += flatten(item) else: retval.append(item) @@ -198,7 +198,7 @@ If the `keepxmlentities` parameter is provided and is a truth value, the core XML entities (&, ', >, < and ") are left intact. - + >>> stripentities('1 < 2 …', keepxmlentities=True) u'1 < 2 \u2026' """ @@ -223,7 +223,7 @@ return ref return _STRIPENTITIES_RE.sub(_replace_entity, text) -_STRIPTAGS_RE = re.compile(r'<[^>]*?>') +_STRIPTAGS_RE = re.compile(r'(<!--.*?-->|<[^>]*>)') def striptags(text): """Return a copy of the text with any XML/HTML tags removed. @@ -234,6 +234,11 @@ >>> striptags('Foo<br />') 'Foo' + HTML/XML comments are stripped, too: + + >>> striptags('<!-- <blub>hehe</blah> -->test') + 'test' + :param text: the string to remove tags from :return: the text with tags removed """
new file mode 100755 --- /dev/null +++ b/scripts/ast_generator.py @@ -0,0 +1,46 @@ +#!/usr/bin/env python2.5 + +"""Script to that automatically generates genshi/templates/_astpy24.py. +Be sure to run this with a Python 2.5 interpreter. +""" + +import _ast + +done = set() + +IGNORE_ATTRS = ('__module__', '__dict__', '__weakref__', '__setattr__', + '__new__', '__getattribute__', '__reduce__', '__delattr__', + '__init__') + +def print_class(cls): + bnames = [] + for base in cls.__bases__: + if base.__module__ == '_ast': + if base not in done: + print_class(base) + bnames.append(base.__name__) + elif base.__module__ == '__builtin__': + bnames.append("%s"%base.__name__) + else: + bnames.append("%s.%s"%(base.__module__,base.__name__)) + print "class %s(%s):"%(cls.__name__, ", ".join(bnames)) + written = False + for attr in cls.__dict__: + if attr not in IGNORE_ATTRS: + written = True + print "\t%s = %s"%(attr, repr(cls.__dict__[attr]),) + if not written: + print "\tpass" + done.add(cls) + +print "# Generated automatically, please do not edit" +print "# Generator can be found in Genshi SVN, scripts/ast-generator.py" +print +print "__version__ = %s" % _ast.__version__ +print + +for name in dir(_ast): + cls = getattr(_ast, name) + if cls.__class__ is type: + print_class(cls) + print
--- a/setup.py +++ b/setup.py @@ -1,7 +1,7 @@ #!/usr/bin/env python # -*- coding: utf-8 -*- # -# Copyright (C) 2006 Edgewall Software +# Copyright (C) 2006-2008 Edgewall Software # All rights reserved. # # This software is licensed as described in the file COPYING, which @@ -13,76 +13,82 @@ # history and logs, available at http://genshi.edgewall.org/log/. from distutils.cmd import Command +from distutils.command.build_ext import build_ext +from distutils.errors import CCompilerError, DistutilsPlatformError import doctest from glob import glob import os try: - from setuptools import setup + from setuptools import setup, Extension, Feature + from setuptools.command.bdist_egg import bdist_egg except ImportError: - from distutils.core import setup + from distutils.core import setup, Extension + Feature = None + bdist_egg = None import sys +sys.path.append(os.path.join('doc', 'common')) +try: + from doctools import build_doc, test_doc +except ImportError: + build_doc = test_doc = None + +_speedup_available = False + +class optional_build_ext(build_ext): + # This class allows C extension building to fail. + def run(self): + try: + build_ext.run(self) + except DistutilsPlatformError, e: + self._unavailable(e) + + def build_extension(self, ext): + try: + build_ext.build_extension(self, ext) + global _speedup_available + _speedup_available = True + except CCompilerError, e: + self._unavailable(e) + + def _unavailable(self, exc): + print '*' * 70 + print """WARNING: +An optional C extension could not be compiled, speedups will not be +available.""" + print '*' * 70 + print exc + -class build_doc(Command): - description = 'Builds the documentation' - user_options = [] - - def initialize_options(self): - pass - - def finalize_options(self): - pass - - def run(self): - from docutils.core import publish_cmdline - docutils_conf = os.path.join('doc', 'docutils.conf') - epydoc_conf = os.path.join('doc', 'epydoc.conf') - - for source in glob('doc/*.txt'): - dest = os.path.splitext(source)[0] + '.html' - if not os.path.exists(dest) or \ - os.path.getmtime(dest) < os.path.getmtime(source): - print 'building documentation file %s' % dest - publish_cmdline(writer_name='html', - argv=['--config=%s' % docutils_conf, source, - dest]) - - try: - from epydoc import cli - old_argv = sys.argv[1:] - sys.argv[1:] = [ - '--config=%s' % epydoc_conf, - '--no-private', # epydoc bug, not read from config - '--simple-term', - '--verbose' - ] - cli.cli() - sys.argv[1:] = old_argv - - except ImportError: - print 'epydoc not installed, skipping API documentation.' +if Feature: + speedups = Feature( + "optionial C speed-enhancements", + standard = True, + ext_modules = [ + Extension('genshi._speedups', ['genshi/_speedups.c']), + ], + ) +else: + speedups = None -class test_doc(Command): - description = 'Tests the code examples in the documentation' - user_options = [] - - def initialize_options(self): - pass +# Setuptools need some help figuring out if the egg is "zip_safe" or not +if bdist_egg: + class my_bdist_egg(bdist_egg): + def zip_safe(self): + return not _speedup_available and bdist_egg.zip_safe(self) - def finalize_options(self): - pass - def run(self): - for filename in glob('doc/*.txt'): - print 'testing documentation file %s' % filename - doctest.testfile(filename, False, optionflags=doctest.ELLIPSIS) +cmdclass = {'build_doc': build_doc, 'test_doc': test_doc, + 'build_ext': optional_build_ext} +if bdist_egg: + cmdclass['bdist_egg'] = my_bdist_egg setup( name = 'Genshi', - version = '0.5', - description = 'A toolkit for stream-based generation of output for the web', + version = '0.6', + description = 'A toolkit for generation of output for the web', long_description = \ """Genshi is a Python library that provides an integrated set of components for parsing, generating, and processing HTML, XML or @@ -93,7 +99,6 @@ license = 'BSD', url = 'http://genshi.edgewall.org/', download_url = 'http://genshi.edgewall.org/wiki/Download', - zip_safe = True, classifiers = [ 'Development Status :: 4 - Beta', @@ -111,13 +116,20 @@ packages = ['genshi', 'genshi.filters', 'genshi.template'], test_suite = 'genshi.tests.suite', - extras_require = {'plugin': ['setuptools>=0.6a2']}, + extras_require = { + 'i18n': ['Babel>=0.8'], + 'plugin': ['setuptools>=0.6a2'] + }, entry_points = """ + [babel.extractors] + genshi = genshi.filters.i18n:extract[i18n] + [python.templating.engines] genshi = genshi.template.plugin:MarkupTemplateEnginePlugin[plugin] genshi-markup = genshi.template.plugin:MarkupTemplateEnginePlugin[plugin] genshi-text = genshi.template.plugin:TextTemplateEnginePlugin[plugin] """, - cmdclass = {'build_doc': build_doc, 'test_doc': test_doc} + features = {'speedups': speedups}, + cmdclass = cmdclass )