changeset 720:acf7c5ee36e7 experimental-newctxt

newctxt branch: Merged revisions [678:835] via svnmerge from [source:trunk].
author cmlenz
date Fri, 11 Apr 2008 08:42:11 +0000
parents 837786a584d5
children
files ChangeLog INSTALL.txt README.txt UPGRADE.txt doc/conf/docutils.ini doc/conf/epydoc.ini doc/filters.txt doc/i18n.txt doc/index.txt doc/install.txt doc/plugin.txt doc/style/bkgnd_pattern.png doc/style/docutils.css doc/style/edgewall.css doc/style/epydoc.css doc/style/pygments.css doc/style/shadow.gif doc/style/vertbars.png doc/templates.txt doc/text-templates.txt doc/upgrade.txt doc/xml-templates.txt examples/bench/basic.py examples/bench/bigtable.py examples/bench/genshi/base.html examples/bench/genshi_text/footer.txt examples/bench/genshi_text/header.txt examples/bench/genshi_text/template.txt examples/cherrypy/config.txt examples/cherrypy/index.html examples/cherrypy/index.py examples/includes/common/macros.html examples/includes/module/test.html examples/includes/run.py examples/includes/skins/default/footer.html examples/includes/skins/default/header.html examples/includes/skins/default/layout.html examples/transform/README.txt examples/transform/index.html examples/transform/run.py examples/transform/template.xml examples/tutorial/geddit/__init__.py examples/tutorial/geddit/controller.py examples/tutorial/geddit/form.py examples/tutorial/geddit/lib/__init__.py examples/tutorial/geddit/lib/ajax.py examples/tutorial/geddit/lib/template.py examples/tutorial/geddit/model.py examples/tutorial/geddit/static/jquery.js examples/tutorial/geddit/static/layout.css examples/tutorial/geddit/static/logo.gif examples/tutorial/geddit/templates/_comment.html examples/tutorial/geddit/templates/_form.html examples/tutorial/geddit/templates/comment.html examples/tutorial/geddit/templates/index.html examples/tutorial/geddit/templates/index.xml examples/tutorial/geddit/templates/info.html examples/tutorial/geddit/templates/info.xml examples/tutorial/geddit/templates/layout.html examples/tutorial/geddit/templates/submit.html genshi/__init__.py genshi/_speedups.c genshi/core.py genshi/filters/html.py genshi/filters/i18n.py genshi/filters/tests/__init__.py genshi/filters/tests/html.py genshi/filters/tests/i18n.py genshi/filters/tests/transform.py genshi/filters/transform.py genshi/output.py genshi/path.py genshi/template/__init__.py genshi/template/base.py genshi/template/directives.py genshi/template/eval.py genshi/template/interpolation.py genshi/template/loader.py genshi/template/markup.py genshi/template/plugin.py genshi/template/tests/directives.py genshi/template/tests/eval.py genshi/template/tests/interpolation.py genshi/template/tests/loader.py genshi/template/tests/markup.py genshi/template/tests/plugin.py genshi/template/tests/templates/new_syntax.txt genshi/template/tests/templates/test_no_doctype.html genshi/template/tests/text.py genshi/template/text.py genshi/tests/core.py genshi/tests/output.py genshi/tests/path.py genshi/util.py setup.py
diffstat 95 files changed, 3556 insertions(+), 1644 deletions(-) [+]
line wrap: on
line diff
--- a/ChangeLog
+++ b/ChangeLog
@@ -17,11 +17,90 @@
    only if the template loader is not set to do automatic reloading. Included
    templates are basically inlined into the including template, which can
    speed up rendering of that template a bit.
+ * Added new syntax for text templates, which is more powerful and flexible
+   with respect to white-space and line breaks. It also supports Python code
+   blocks. The old syntax is still available and the default for now, but in a
+   future release the new syntax will become the default, and some time after
+   that the old syntax will be removed.
+ * Added support for passing optimization hints to `<py:match>` directives,
+   which can speed up match templates in many cases, for example when a match
+   template should only be applied once to a stream, or when it should not be
+   applied recursively.
+ * Text templates now default to rendering as plain text; it is no longer
+   necessary to explicitly specify the "text" method to the `render()` or
+   `serialize()` method of the generated markup stream.
+ * XInclude elements in markup templates now support the `parse` attribute;
+   when set to "xml" (the default), the include is processed as before, but
+   when set to "text", the included template is parsed as a text template using
+   the new syntax (ticket #101).
+ * Python code blocks inside match templates are now executed (ticket #155).
+ * The template engine plugin no longer adds the `default_doctype` when the
+   `fragment` parameter is `True`.
+ * The `striptags` function now also removes HTML/XML-style comments (ticket
+   #150).
+ * The `py:replace` directive can now also be used as an element, with an
+   attribute named `value` (ticket #144).
+ * The `TextSerializer` class no longer strips all markup in text by default,
+   so that it is still possible to use the Genshi `escape` function even with
+   text templates. The old behavior is available via the `strip_markup` option
+   of the serializer (ticket #146).
+ * Assigning to a variable named `data` in a Python code block no longer
+   breaks context lookup.
+ * The `Stream.render` now accepts an optional `out` parameter that can be
+   used to pass in a writable file-like object to use for assembling the
+   output, instead of building a big string and returning it.
+ * The XHTML serializer now strips `xml:space` attributes as they are only
+   allowed on very few tags.
+ * Match templates are now applied in a more controlled fashion: in the order
+   they are declared in the template source, all match templates up to (and
+   including) the matching template itself are applied to the matched content,
+   whereas the match templates declared after the matching template are only
+   applied to the generated content (ticket #186).
+ * The `TemplateLoader` class now provides an `_instantiate()` method that can
+   be overridden by subclasses to implement advanced template instantiation
+   logic (ticket #204).
+ * The search path of the `TemplateLoader` class can now contain ''load
+   functions'' in addition to path strings. A load function is passed the
+   name of the requested template file, and should return a file-like object
+   and some metadata. New load functions are supplied for loading from egg
+   package data, and loading from different loaders depending on the path
+   prefix of the requested filename (ticket #182).
+ * Match templates can now be processed without keeping the complete matched
+   content in memory, which could cause excessive memory use on long pages.
+   The buffering can be disabled using the new `buffer` optimization hint on
+   the `<py:match>` directive.
+ * Improve error reporting when accessing an attribute in a Python expression
+   raises an `AttributeError` (ticket #191).
+ * The `Markup` class now supports mappings for right hand of the `%` (modulo)
+   operator in the same way the Python string classes do, except that the
+   substituted values are escape. Also, the special constructor which took
+   positional arguments that would be substituted was removed. Thus the
+   `Markup` class now supports the same arguments as that of its `unicode`
+   base class (ticket #211).
+ * The `Template` class and its subclasses, as well as the interpolation API,
+   now take an `filepath` parameter instead of `basedir` (ticket #207).
+
+
+Version 0.4.4
+http://svn.edgewall.org/repos/genshi/tags/0.4.4/
+(Aug 14, 2007, from branches/stable/0.4.x)
+
+ * Fixed augmented assignment to local variables in Python code blocks.
+ * Fixed handling of nested function and class definitions in Python code
+   blocks.
+ * Includes were not raising `TemplateNotFound` exceptions even when no
+   fallback has been specified. That has been corrected.
+ * The template loader now raises a `TemplateNotFound` error when a previously
+   cached template is removed or renamed, where it previously was passing up
+   an `OSError`.
+ * The Genshi I18n filter can be configured to only extract messages found in
+   `gettext` function calls, ignoring any text nodes and attribute values
+   (ticket #138).
 
 
 Version 0.4.3
 http://svn.edgewall.org/repos/genshi/tags/0.4.3/
-(?, from branches/stable/0.4.x)
+(Jul 17 2007, from branches/stable/0.4.x)
 
  * The I18n filter no longer extracts or translates literal strings in
    attribute values that also contain expressions.
@@ -34,11 +113,16 @@
    ignored tags (ticket #132).
  * The HTML sanitizer now strips any CSS comments in style attributes, which
    could previously be used to hide malicious property values.
+ * The HTML sanitizer now also removes any HTML comments encountered, as those
+   may be used to hide malicious payloads targetting a certain "innovative"
+   browser that goes and interprets the content of specially prepared comments.
+ * Attribute access in template expressions no longer silently ignores
+   exceptions other than `AttributeError` raised in the attribute accessor.
 
 
 Version 0.4.2
 http://svn.edgewall.org/repos/genshi/tags/0.4.2/
-(Jun 20, from branches/stable/0.4.x)
+(Jun 20 2007, from branches/stable/0.4.x)
 
  * The `doctype` parameter of the markup serializers now also accepts the "name"
    of the doctype as string, in addition to the `(name, pubid, sysid)` tuple.
deleted file mode 100644
--- a/INSTALL.txt
+++ /dev/null
@@ -1,37 +0,0 @@
-Installing Genshi
-=================
-
-Prerequisites
--------------
-
- * Python 2.3 or later (2.4 or later is recommended)
- * Optional: setuptools 0.6a2 or later
-
-
-Installation
-------------
-
-Once you've downloaded and unpacked a Genshi source release, enter the
-directory where the archive was unpacked, and run:
-
-  $ python setup.py install
-
-Note that you may need administrator/root privileges for this step, as
-this command will by default attempt to install Genshi to the Python
-site-packages directory on your system.
-
-For advanced options, please refer to the easy_install and/or the distutils
-documentation:
-
-  http://peak.telecommunity.com/DevCenter/EasyInstall
-  http://docs.python.org/inst/inst.html
-
-
-Support
--------
-
-If you encounter any problems with Genshi, please don't hesitate to ask
-questions on the Genshi mailing list or IRC channel:
-
-  http://genshi.edgewall.org/wiki/MailingList
-  http://genshi.edgewall.org/wiki/IrcChannel
--- a/README.txt
+++ b/README.txt
@@ -1,11 +1,12 @@
 About Genshi
 ============
 
-Genshi is a Python library that provides an integrated set of components
-for parsing, generating, and processing HTML, XML or other textual
-content for output generation on the web. The major feature is a
-template language, which is heavily inspired by Kid.
+Genshi is a Python library that provides an integrated set of
+components for parsing, generating, and processing HTML, XML or other
+textual content for output generation on the web. The major feature is
+a template language, which is heavily inspired by Kid.
 
-For more information please visit the Genshi web site:
+For more information please see the documentation in the `doc`
+directory, and visit the Genshi web site:
 
   <http://genshi.edgewall.org/>
deleted file mode 100644
--- a/UPGRADE.txt
+++ /dev/null
@@ -1,51 +0,0 @@
-Upgrading Genshi
-================
-
-Upgrading from Genshi 0.3.x to 0.4.x
-------------------------------------
-
-The modules ``genshi.filters`` and ``genshi.template`` have been
-refactored into packages containing multiple modules. While code using
-the regular APIs should continue to work without problems, you should
-make sure to remove any leftover traces of the ``template.py`` file on
-the installation path. This is not necessary when Genshi was installed
-as a Python egg.
-
-Results of evaluating template expressions are no longer implicitly
-called if they are callable. If you have been using that feature, you
-will need to add the parenthesis to actually call the function.
-
-Instances of `genshi.core.Attrs` are now immutable. Filters
-manipulating the attributes in a stream may need to be updated. Also,
-the `Attrs` class no longer automatically wraps all attribute names
-in `QName` objects, so users of the `Attrs` class need to do this
-themselves. See the documentation of the `Attrs` class for more
-information.
-
-
-Upgrading from Markup
----------------------
-
-Prior to version 0.3, the name of the Genshi project was "Markup". The
-name change means that you will have to adjust your import statements
-and the namespace URI of XML templates, among other things:
-
- * The package name was changed from "markup" to "genshi". Please
-   adjust any import statements referring to the old package name.
- * The namespace URI for directives in Genshi XML templates has changed
-   from http://markup.edgewall.org/ to http://genshi.edgewall.org/.
-   Please update the xmlns:py declaration in your template files
-   accordingly.
-
-Furthermore, due to the inclusion of a text-based template language,
-the class:
-
-  `markup.template.Template`
-
-has been renamed to:
-
-  `genshi.template.MarkupTemplate`
-
-If you've been using the Template class directly, you'll need to
-update your code (a simple find/replace should do--the API itself
-did not change).
deleted file mode 100644
--- a/doc/conf/docutils.ini
+++ /dev/null
@@ -1,9 +0,0 @@
-[general]
-input_encoding = utf-8
-strip_comments = yes
-toc_backlinks = none
-
-[html4css1 writer]
-embed_stylesheet = no
-stylesheet = style/edgewall.css
-xml_declaration = no
deleted file mode 100644
--- a/doc/conf/epydoc.ini
+++ /dev/null
@@ -1,24 +0,0 @@
-[epydoc]
-
-name: Documentation Index
-url: ../index.html
-modules: genshi
-verbosity: 1
-
-# Extraction
-docformat: restructuredtext
-parse: yes
-introspect: yes
-exclude: .*\.tests.*
-inheritance: listed
-private: no
-imports: no
-include-log: no
-
-# HTML output
-output: html
-target: doc/api/
-css: doc/style/epydoc.css
-top: genshi
-frames: no
-sourcecode: no
--- a/doc/filters.txt
+++ b/doc/filters.txt
@@ -19,7 +19,7 @@
 ================
 
 The filter ``genshi.filters.html.HTMLFormFiller`` can automatically populate an
-HTML form from values provided as a simple dictionary. When using thi filter,
+HTML form from values provided as a simple dictionary. When using this filter,
 you can basically omit any ``value``, ``selected``, or ``checked`` attributes
 from form controls in your templates, and let the filter do all that work for
 you.
--- a/doc/i18n.txt
+++ b/doc/i18n.txt
@@ -145,20 +145,31 @@
 .. _`text templates`: text-templates.html
 
 ``encoding``
-------------------
+------------
 The encoding of the template file. This is only used for text templates. The
 default is to assume “utf-8”.
 
 ``include_attrs``
-------------------
+-----------------
 Comma-separated list of attribute names that should be considered to have 
 localizable values. Only used for markup templates.
 
-``include_tags``
-------------------
+``ignore_tags``
+---------------
 Comma-separated list of tag names that should be ignored. Only used for markup 
 templates.
 
+``extract_text``
+----------------
+Whether text outside explicit ``gettext`` function calls should be extracted.
+By default, any text nodes not inside ignored tags, and values of attribute in
+the ``include_attrs`` list are extracted. If this option is disabled, only 
+strings in ``gettext`` function calls are extracted.
+
+.. note:: If you disable this option, it's not necessary to add the translation
+          filter as described above. You only need to make sure that the
+          template has access to the ``gettext`` functions it uses.
+
 
 Translation
 ===========
--- a/doc/index.txt
+++ b/doc/index.txt
@@ -1,8 +1,8 @@
 .. -*- mode: rst; encoding: utf-8 -*-
 
-======
-Genshi
-======
+=======
+Preface
+=======
 
 .. image:: logo.png
    :width: 225
@@ -11,15 +11,24 @@
    :alt: Genshi - Generate output for the web
    :class: logo
 
----------------------------------------------------------
-Toolkit for stream-based generation of output for the web
----------------------------------------------------------
+--------------------------------------------
+Toolkit for generation of output for the web
+--------------------------------------------
 
 Genshi is a Python library that provides an integrated set of components
 for parsing, generating, and processing HTML, XML or other textual content
 for output generation on the web. The major feature is a template language,
 which is heavily inspired by Kid.
 
+Installation
+------------
+
+* `Installing Genshi <install.html>`_
+* `Upgrading from Previous Versions <upgrade.html>`_
+
+Usage
+-----
+
 * `Markup Streams <streams.html>`_
 * `Templating Basics <templates.html>`_
 * `XML Template Language <xml-templates.html>`_
@@ -28,4 +37,8 @@
 * `Using XPath <xpath.html>`_
 * `Internationalization and Localization <i18n.html>`_
 * `Using the Templating Plugin <plugin.html>`_
+
+API Documentation
+-----------------
+
 * `Generated API Documentation <api/index.html>`_
new file mode 100644
--- /dev/null
+++ b/doc/install.txt
@@ -0,0 +1,88 @@
+Installing Genshi
+=================
+
+
+.. contents:: Contents
+   :depth: 2
+.. sectnum::
+
+
+Prerequisites
+-------------
+
+* Python_ 2.3 or later (2.4 or later is strongly recommended)
+* Optional: Setuptools_ 0.6a2 or later
+
+.. _python: http://www.python.org/
+.. _setuptools: http://cheeseshop.python.org/pypi/setuptools
+
+Setuptools is only required for the `template engine plugin`_, which can be
+used to integrate Genshi with Python web application frameworks such as Pylons
+or TurboGears. Genshi also provides a Setuptools-based plugin that integrates
+its `internationalization support`_ with the Babel_ library, but that support
+can also be used without Setuptools being available (although in a slightly
+less convenient fashion).
+
+.. _`template engine plugin`: plugin.html
+.. _`internationalization support`: i18n.html
+.. _babel: http://babel.edgewall.org/
+
+
+Installing via ``easy_install``
+-------------------------------
+
+If you have a recent version of Setuptools_ installed, you can directly install
+Genshi using the easy_install command-line tool::
+
+  $ easy_install Genshi
+
+This downloads and installs the latest version of the Genshi package.
+
+If you have an older Genshi release installed and would like to upgrade, add
+the ``-U`` option to the above command.
+
+
+Installing from a Binary Installer
+----------------------------------
+
+Binary packages for Windows and Mac OS X are provided for Genshi. To install
+from such a package, simply download and open it.
+
+
+Installing from a Source Tarball
+--------------------------------
+
+Once you've downloaded and unpacked a Genshi source release, enter the
+directory where the archive was unpacked, and run::
+
+  $ python setup.py install
+
+Note that you may need administrator/root privileges for this step, as this
+command will by default attempt to install Genshi to the Python
+``site-packages`` directory on your system.
+
+Genshi comes with an optional extension module written in C that is used to
+improve performance in some areas. This extension is automatically compiled
+when you run the ``setup.py`` script as shown above. In the case that the
+extension can not be compiled, possibly due to a missing or incompatible C
+compiler, the compilation is skipped. If you'd prefer Genshi to not use this
+native extension module, you can explicitly bypass the compilation using the
+``--without-speedups`` option::
+
+  $ python setup.py --without-speedups install
+
+For other build and installation options, please consult the easy_install_
+and/or the Python distutils_ documentation.
+
+.. _easy_install: http://peak.telecommunity.com/DevCenter/EasyInstall
+.. _distutils:  http://docs.python.org/inst/inst.html
+
+
+Support
+-------
+
+If you encounter any problems with Genshi, please don't hesitate to ask
+questions on the Genshi `mailing list`_ or `IRC channel`_.
+
+.. _`mailing list`: http://genshi.edgewall.org/wiki/MailingList
+.. _`irc channel`: http://genshi.edgewall.org/wiki/IrcChannel
--- a/doc/plugin.txt
+++ b/doc/plugin.txt
@@ -222,6 +222,14 @@
 The default value is **25**. You may want to choose a higher value if your web
 site uses a larger number of templates, and you have enough memory to spare.
 
+``genshi.new_text_syntax``
+--------------------------
+Whether the new syntax for text templates should be used. Specify "yes" to
+enable the new syntax, or "no" to use the old syntax.
+
+In the version of Genshi, the default is to use the old syntax for
+backwards-compatibility, but that will change in a future release.
+
 ``genshi.search_path``
 ----------------------
 A colon-separated list of file-system path names that the template loader should
deleted file mode 100644
index 90e92682135d3f7213332f870f973bd06d6d57ee..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
literal 0
Hc$@<O00001
deleted file mode 100644
--- a/doc/style/docutils.css
+++ /dev/null
@@ -1,277 +0,0 @@
-/*
-:Author: David Goodger
-:Contact: goodger@users.sourceforge.net
-:Date: $Date: 2005-12-18 01:56:14 +0100 (Sun, 18 Dec 2005) $
-:Revision: $Revision: 4224 $
-:Copyright: This stylesheet has been placed in the public domain.
-
-Default cascading style sheet for the HTML output of Docutils.
-
-See http://docutils.sf.net/docs/howto/html-stylesheets.html for how to
-customize this style sheet.
-*/
-
-/* used to remove borders from tables and images */
-.borderless, table.borderless td, table.borderless th {
-  border: 0 }
-
-table.borderless td, table.borderless th {
-  /* Override padding for "table.docutils td" with "! important".
-     The right padding separates the table cells. */
-  padding: 0 0.5em 0 0 ! important }
-
-.first {
-  /* Override more specific margin styles with "! important". */
-  margin-top: 0 ! important }
-
-.last, .with-subtitle {
-  margin-bottom: 0 ! important }
-
-.hidden {
-  display: none }
-
-a.toc-backref {
-  text-decoration: none ;
-  color: black }
-
-blockquote.epigraph {
-  margin: 2em 5em ; }
-
-dl.docutils dd {
-  margin-bottom: 0.5em }
-
-dl.docutils dt {
-  font-weight: bold }
-
-div.abstract {
-  margin: 2em 5em }
-
-div.abstract p.topic-title {
-  font-weight: bold ;
-  text-align: center }
-
-div.admonition, div.attention, div.caution, div.danger, div.error,
-div.hint, div.important, div.note, div.tip, div.warning {
-  margin: 2em ;
-  border: medium outset ;
-  padding: 1em }
-
-div.admonition p.admonition-title, div.hint p.admonition-title,
-div.important p.admonition-title, div.note p.admonition-title,
-div.tip p.admonition-title {
-  font-weight: bold ;
-  font-family: sans-serif }
-
-div.attention p.admonition-title, div.caution p.admonition-title,
-div.danger p.admonition-title, div.error p.admonition-title,
-div.warning p.admonition-title {
-  color: red ;
-  font-weight: bold ;
-  font-family: sans-serif }
-
-/* Uncomment (and remove this text!) to get reduced vertical space in
-   compound paragraphs.
-div.compound .compound-first, div.compound .compound-middle {
-  margin-bottom: 0.5em }
-
-div.compound .compound-last, div.compound .compound-middle {
-  margin-top: 0.5em }
-*/
-
-div.dedication {
-  margin: 2em 5em ;
-  text-align: center ;
-  font-style: italic }
-
-div.dedication p.topic-title {
-  font-weight: bold ;
-  font-style: normal }
-
-div.figure {
-  margin-left: 2em ;
-  margin-right: 2em }
-
-div.footer, div.header {
-  clear: both;
-  font-size: smaller }
-
-div.line-block {
-  display: block ;
-  margin-top: 1em ;
-  margin-bottom: 1em }
-
-div.line-block div.line-block {
-  margin-top: 0 ;
-  margin-bottom: 0 ;
-  margin-left: 1.5em }
-
-div.sidebar {
-  margin-left: 1em ;
-  border: medium outset ;
-  padding: 1em ;
-  background-color: #ffffee ;
-  width: 40% ;
-  float: right ;
-  clear: right }
-
-div.sidebar p.rubric {
-  font-family: sans-serif ;
-  font-size: medium }
-
-div.system-messages {
-  margin: 5em }
-
-div.system-messages h1 {
-  color: red }
-
-div.system-message {
-  border: medium outset ;
-  padding: 1em }
-
-div.system-message p.system-message-title {
-  color: red ;
-  font-weight: bold }
-
-div.topic {
-  margin: 2em }
-
-h1.section-subtitle, h2.section-subtitle, h3.section-subtitle,
-h4.section-subtitle, h5.section-subtitle, h6.section-subtitle {
-  margin-top: 0.4em }
-
-h1.title {
-  text-align: center }
-
-h2.subtitle {
-  text-align: center }
-
-hr.docutils {
-  width: 75% }
-
-img.align-left {
-  clear: left }
-
-img.align-right {
-  clear: right }
-
-ol.simple, ul.simple {
-  margin-bottom: 1em }
-
-ol.arabic {
-  list-style: decimal }
-
-ol.loweralpha {
-  list-style: lower-alpha }
-
-ol.upperalpha {
-  list-style: upper-alpha }
-
-ol.lowerroman {
-  list-style: lower-roman }
-
-ol.upperroman {
-  list-style: upper-roman }
-
-p.attribution {
-  text-align: right ;
-  margin-left: 50% }
-
-p.caption {
-  font-style: italic }
-
-p.credits {
-  font-style: italic ;
-  font-size: smaller }
-
-p.label {
-  white-space: nowrap }
-
-p.rubric {
-  font-weight: bold ;
-  font-size: larger ;
-  color: maroon ;
-  text-align: center }
-
-p.sidebar-title {
-  font-family: sans-serif ;
-  font-weight: bold ;
-  font-size: larger }
-
-p.sidebar-subtitle {
-  font-family: sans-serif ;
-  font-weight: bold }
-
-p.topic-title {
-  font-weight: bold }
-
-pre.address {
-  margin-bottom: 0 ;
-  margin-top: 0 ;
-  font-family: serif ;
-  font-size: 100% }
-
-pre.literal-block, pre.doctest-block {
-  margin-left: 2em ;
-  margin-right: 2em ;
-  background-color: #eeeeee }
-
-span.classifier {
-  font-family: sans-serif ;
-  font-style: oblique }
-
-span.classifier-delimiter {
-  font-family: sans-serif ;
-  font-weight: bold }
-
-span.interpreted {
-  font-family: sans-serif }
-
-span.option {
-  white-space: nowrap }
-
-span.pre {
-  white-space: pre }
-
-span.problematic {
-  color: red }
-
-span.section-subtitle {
-  /* font-size relative to parent (h1..h6 element) */
-  font-size: 80% }
-
-table.citation {
-  border-left: solid 1px gray;
-  margin-left: 1px }
-
-table.docinfo {
-  margin: 2em 4em }
-
-table.docutils {
-  margin-top: 0.5em ;
-  margin-bottom: 0.5em }
-
-table.footnote {
-  border-left: solid 1px black;
-  margin-left: 1px }
-
-table.docutils td, table.docutils th,
-table.docinfo td, table.docinfo th {
-  padding-left: 0.5em ;
-  padding-right: 0.5em ;
-  vertical-align: top }
-
-table.docutils th.field-name, table.docinfo th.docinfo-name {
-  font-weight: bold ;
-  text-align: left ;
-  white-space: nowrap ;
-  padding-left: 0 }
-
-h1 tt.docutils, h2 tt.docutils, h3 tt.docutils,
-h4 tt.docutils, h5 tt.docutils, h6 tt.docutils {
-  font-size: 100% }
-
-tt.docutils {
-  background-color: #eeeeee }
-
-ul.auto-toc {
-  list-style-type: none }
deleted file mode 100644
--- a/doc/style/edgewall.css
+++ /dev/null
@@ -1,77 +0,0 @@
-@import url(docutils.css);
-@import url(pygments.css);
-
-html, body { height: 100%; margin: 0; padding: 0; }
-html, body { background: #4b4d4d url(bkgnd_pattern.png); color: #000; }
-body, th, td {
-  font: normal small Verdana,Arial,'Bitstream Vera Sans',Helvetica,sans-serif;
-}
-pre, code, tt { font-size: medium; }
-h1, h2, h3, h4 {
-  border-bottom: 1px solid #ccc;
-  font-family: Arial,Verdana,'Bitstream Vera Sans',Helvetica,sans-serif;
-  font-weight: bold; letter-spacing: -0.018em;
-}
-h1 { font-size: 19px; margin: 2em 0 .5em; }
-h2 { font-size: 16px; margin: 1.5em 0 .5em; }
-h3 { font-size: 14px; margin: 1.2em 0 .5em; }
-hr { border: none;  border-top: 1px solid #ccb; margin: 2em 0; }
-p { margin: 0 0 1em; }
-
-table { border: 1px solid #999; border-width: 0 1px 0 0;
-  border-collapse: separate; border-spacing: 0;
-}
-table thead th { background: #999; border: 1px solid #999;; color: #fff;
-  font-weight: bold;
-}
-table td { border: 1px solid #ccc; border-width: 0 0 1px 1px; padding: .3em; }
-
-:link, :visited { text-decoration: none; border-bottom: 1px dotted #bbb;
-  color: #b00;
-}
-:link:hover, :visited:hover { background-color: #eee; color: #555; }
-:link img, :visited img { border: none }
-h1 :link, h1 :visited ,h2 :link, h2 :visited, h3 :link, h3 :visited,
-h4 :link, h4 :visited, h5 :link, h5 :visited, h6 :link, h6 :visited {
-  color: #000;
-}
-
-div.document { background: #fff url(shadow.gif) right top repeat-y;
-  border-left: 1px solid #000; margin: 0 auto 0 40px;
-  min-height: 100%; width: 54em; padding: 0 180px 1px 20px;
-}
-h1.title, div.document#genshi h1 { border: none; color: #666;
-  font-size: x-large; margin: 0 -20px 1em; padding: 2em 20px 0;
-}
-h1.title { background: url(vertbars.png) repeat-x; }
-div.document#genshi h1.title { text-indent: -4000px; }
-div.document#genshi h1 { text-align: center; }
-pre.literal-block, div.highlight pre { background: #f4f4f4;
-  border: 1px solid #e6e6e6; color: #000; margin: 1em 1em; padding: .25em;
-  overflow: auto;
-}
-
-div.contents { font-size: 90%; position: absolute; position: fixed;
-  margin-left: 80px; left: 60em; top: 30px; right: 0;
-}
-div.contents .topic-title { display: none; }
-div.contents ul { list-style: none; padding-left: 0; }
-div.contents :link, div.contents :visited { color: #c6c6c6; border: none;
-  display: block; padding: 3px 5px 3px 10px;
-}
-div.contents :link:hover, div.contents :visited:hover { background: #000;
-  color: #fff;
-}
-
-div.admonition, div.attention, div.caution, div.danger, div.error, div.hint,
-div.important, div.note, div.tip, div.warning {
-  border: none; color: #333; font-style: italic; margin: 1em 2em;
-}
-div.attention p.admonition-title, div.caution p.admonition-title, div.danger
-p.admonition-title, div.error p.admonition-title,
-div.warning p.admonition-title {
-  color: #b00;
-}
-p.admonition-title { margin-bottom: 0; text-transform: uppercase; }
-tt.docutils { background-color: transparent; }
-span.pre { white-space: normal }
deleted file mode 100644
--- a/doc/style/epydoc.css
+++ /dev/null
@@ -1,136 +0,0 @@
-html { background: #4b4d4d url(../style/bkgnd_pattern.png); margin: 0;
-  padding: 1em 1em 3em;
-}
-body { background: #fff url(../style/vertbars.png) repeat-x;
-  border: 1px solid #000; color: #000; margin: 1em 0; padding: 0 1em 1em;
-}
-body, th, td {
-  font: normal small Verdana,Arial,'Bitstream Vera Sans',Helvetica,sans-serif;
-}
-h1, h2, h3, h4 {
-  font-family: Arial,Verdana,'Bitstream Vera Sans',Helvetica,sans-serif;
-  font-weight: bold; letter-spacing: -0.018em;
-}
-h1 { font-size: 19px; margin: 2em 0 .5em; }
-h2 { font-size: 16px; margin: 1.5em 0 .5em; }
-h3 { font-size: 14px; margin: 1.2em 0 .5em; }
-hr { border: none;  border-top: 1px solid #ccb; margin: 2em 0; }
-p { margin: 0 0 1em; }
-:link, :visited { text-decoration: none; border-bottom: 1px dotted #bbb;
-  color: #b00;
-}
-:link:hover, :visited:hover { background-color: #eee; color: #555; }
-
-table { border: none; border-collapse: collapse; }
-
-table.navbar { background: #000; color: #fff; margin: 2em 0 .33em; }
-table.navbar th { border: 1px solid #000; font-weight: bold; padding: 1px; }
-table.navbar :link, table.navbar :visited { border: none; color: #fff; }
-table.navbar :link:hover, table.navbar :visited:hover { background: none;
-  text-decoration: underline overline;
-}
-table.navbar th.navbar-select { background: #fff; color: #000; }
-span.breadcrumbs { color: #666; font-size: 95%; }
-h1.epydoc { border: none; color: #666;
-  font-size: x-large; margin: 1em 0 0; padding: 0;
-}
-pre.base-tree { color: #666; margin: 0; padding: 0; }
-pre.base-tree :link, pre.base-tree :visited { border: none; }
-pre.py-doctest, pre.variable, pre.rst-literal-block { background: #eee;
-  border: 1px solid #e6e6e6; color: #000; margin: 1em; padding: .25em;
-  overflow: auto;
-}
-pre.variable { margin: 0; }
-
-/* Summary tables */
-
-table.summary { margin: .5em 0; }
-table.summary tr.table-header { background: #f7f7f0; }
-table.summary td.table-header { color: #666; font-weight: bold; }
-table.summary th.group-header { background: #f7f7f0; color: #666;
-  font-size: 90%; font-weight: bold; text-align: left;
-}
-table.summary th, table.summary td { border: 1px solid #d7d7d7; }
-table.summary th th, table.summary td td { border: none; }
-table.summary td.summary table td { color: #666; font-size: 90%; }
-table.summary td.summary table br { display: none; }
-table.summary td.summary span.summary-type { font-family: monospace;
-  font-size: 90%;
-}
-table.summary td.summary span.summary-type code { font-size: 110%; }
-p.indent-wrapped-lines { color: #999; font-size: 85%; margin: 0;
-  padding: 0 0 0 7em; text-indent: -7em;
-}
-p.indent-wrapped-lines code { color: #999; font-size: 115%; }
-p.indent-wrapped-lines :link, p.indent-wrapped-lines :visited { border: none; }
-.summary-sig { display: block; font-family: monospace; font-size: 120%;
-  margin-bottom: .5em;
-}
-.summary-sig-name { font-weight: bold; }
-.summary-sig-arg { color: #333; }
-.summary-sig :link, .summary-sig :visited { border: none; }
-.summary-name { font-family: monospace; font-weight: bold; }
-
-/* Details tables */
-
-table.details { margin: 2em 0 0; }
-div table.details { margin-top: 0; }
-table.details tr.table-header { background: transparent; }
-table.details td.table-header { border-bottom: 1px solid #ccc; padding: 2em 0 0; }
-table.details span.table-header {
-  font: bold 140% Arial,Verdana,'Bitstream Vera Sans',Helvetica,sans-serif;
-  letter-spacing: -0.018em;
-}
-table.details th, table.details td { border: none; }
-table.details th th, table.details td td { border: none; }
-table.details td { padding-left: 2em; }
-table.details td td { padding-left: 0; }
-table.details h3.epydoc { margin-left: -2em; }
-table.details h3.epydoc .sig { color: #999; font-family: monospace; }
-table.details h3.epydoc .sig-name { color: #000; }
-table.details h3.epydoc .sig-arg { color: #666; }
-table.details h3.epydoc .sig-default { font-size: 95%; font-weight: normal; }
-table.details h3.epydoc .sig-default code { font-weight: normal; }
-table.details h3.epydoc .fname { color: #999; font-size: 90%;
-  font-style: italic; font-weight: normal; line-height: 1.6em;
-}
-
-dl dt { color: #666; margin-top: 1em; }
-dl dd { margin: 0; padding-left: 2em; }
-dl.fields { margin: 1em 0; padding: 0; }
-dl.fields dt { color: #666; margin-top: 1em; }
-dl.fields dd ul { margin: 0; padding: 0; }
-div.fields { font-size: 90%; margin: 0 0 2em 2em; }
-div.fields p { margin-bottom: 0.5em; }
-
-table td.footer { color: #999; font-size: 85%; margin-top: 3em;
-  padding: 0 3em 1em; position: absolute; width: 80%; }
-table td.footer :link, table td.footer :visited { border: none; color: #999; }
-table td.footer :link:hover, table td.footer :visited:hover {
-  background: transparent; text-decoration: underline;
-}
-
-/* Syntax highlighting */
-
-.py-prompt, .py-more, .variable-ellipsis, .variable-op { color: #999; }
-.variable-group { color: #666; font-weight: bold; }
-.py-string, .variable-string, .variable-quote { color: #093; }
-.py-comment { color: #06f; font-style: italic; }
-.py-keyword { color: #00f; }
-.py-output { background: #f6f6f0; color: #666; font-weight: bold; }
-
-/* Index */
-
-table.link-index { background: #f6f6f0; border: none; margin-top: 1em; }
-table.link-index td.link-index { border: none; font-family: monospace;
-  font-weight: bold; padding: .5em 1em;
-}
-table.link-index td table, table.link-index td td { border: none; }
-table.link-index .index-where { color: #999;
-  font-family: Verdana,Arial,'Bitstream Vera Sans',Helvetica,sans-serif;
-  font-size: 90%; font-weight: normal; line-height: 1.6em;
-}
-table.link-index .index-where :link, table.link-index .index-where :visited {
-  border: none; color: #666;
-}
-h2.epydoc { color: #999; font-size: 200%; line-height: 10px; }
deleted file mode 100644
--- a/doc/style/pygments.css
+++ /dev/null
@@ -1,57 +0,0 @@
-div.highlight { background: #ffffff; }
-div.highlight .c { color: #999988; font-style: italic }
-div.highlight .err { color: #a61717; background-color: #e3d2d2 }
-div.highlight .k { font-weight: bold }
-div.highlight .o { font-weight: bold }
-div.highlight .cm { color: #999988; font-style: italic }
-div.highlight .cp { color: #999999; font-weight: bold }
-div.highlight .c1 { color: #999988; font-style: italic }
-div.highlight .cs { color: #999999; font-weight: bold; font-style: italic }
-div.highlight .gd { color: #000000; background-color: #ffdddd }
-div.highlight .ge { font-style: italic }
-div.highlight .gr { color: #aa0000 }
-div.highlight .gh { color: #999999 }
-div.highlight .gi { color: #000000; background-color: #ddffdd }
-div.highlight .go { color: #888888 }
-div.highlight .gp { color: #555555 }
-div.highlight .gs { font-weight: bold }
-div.highlight .gu { color: #aaaaaa }
-div.highlight .gt { color: #aa0000 }
-div.highlight .kc { font-weight: bold }
-div.highlight .kd { font-weight: bold }
-div.highlight .kp { font-weight: bold }
-div.highlight .kr { font-weight: bold }
-div.highlight .kt { color: #445588; font-weight: bold }
-div.highlight .m { color: #009999 }
-div.highlight .s { color: #bb8844 }
-div.highlight .na { color: #008080 }
-div.highlight .nb { color: #999999 }
-div.highlight .nc { color: #445588; font-weight: bold }
-div.highlight .no { color: #008080 }
-div.highlight .ni { color: #800080 }
-div.highlight .ne { color: #990000; font-weight: bold }
-div.highlight .nf { color: #990000; font-weight: bold }
-div.highlight .nn { color: #555555 }
-div.highlight .nt { color: #000080 }
-div.highlight .nv { color: #008080 }
-div.highlight .ow { font-weight: bold }
-div.highlight .mf { color: #009999 }
-div.highlight .mh { color: #009999 }
-div.highlight .mi { color: #009999 }
-div.highlight .mo { color: #009999 }
-div.highlight .sb { color: #bb8844 }
-div.highlight .sc { color: #bb8844 }
-div.highlight .sd { color: #bb8844 }
-div.highlight .s2 { color: #bb8844 }
-div.highlight .se { color: #bb8844 }
-div.highlight .sh { color: #bb8844 }
-div.highlight .si { color: #bb8844 }
-div.highlight .sx { color: #bb8844 }
-div.highlight .sr { color: #808000 }
-div.highlight .s1 { color: #bb8844 }
-div.highlight .ss { color: #bb8844 }
-div.highlight .bp { color: #999999 }
-div.highlight .vc { color: #008080 }
-div.highlight .vg { color: #008080 }
-div.highlight .vi { color: #008080 }
-div.highlight .il { color: #009999 }
deleted file mode 100644
index 326cd1b37dcd637d2e8b7193fe0011f51cca3ce9..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
literal 0
Hc$@<O00001
deleted file mode 100644
index 42ae3f86d7f1513fd585ad02959afbb37ad476cb..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
literal 0
Hc$@<O00001
--- a/doc/templates.txt
+++ b/doc/templates.txt
@@ -116,9 +116,12 @@
   >>> from genshi.template import MarkupTemplate
   >>> tmpl = MarkupTemplate('<h1>Hello, $name!</h1>')
   >>> stream = tmpl.generate(name='world')
-  >>> print stream.render()
+  >>> print stream.render('xhtml')
   <h1>Hello, world!</h1>
 
+.. note:: See the Serialization_ section of the `Markup Streams`_ page for
+          information on configuring template output options.
+
 Using a text template is similar:
 
 .. code-block:: pycon
@@ -126,13 +129,15 @@
   >>> from genshi.template import TextTemplate
   >>> tmpl = TextTemplate('Hello, $name!')
   >>> stream = tmpl.generate(name='world')
-  >>> print stream.render()
+  >>> print stream
   Hello, world!
 
-.. note:: See the Serialization_ section of the `Markup Streams`_ page for
-          information on configuring template output options.
+.. note:: If you want to use text templates, you should consider using the
+          ``NewTextTemplate`` class instead of simply ``TextTemplate``. See
+          the `Text Template Language`_ page.
 
 .. _serialization: streams.html#serialization
+.. _`Text Template Language`: text-templates.html
 .. _`Markup Streams`: streams.html
 
 Using a template loader provides the advantage that “compiled” templates are
@@ -203,8 +208,8 @@
 Code Blocks
 ===========
 
-XML templates also support full Python code blocks using the ``<?python ?>``
-processing instruction:
+Templates also support full Python code blocks, using the ``<?python ?>``
+processing instruction in XML templates:
 
 .. code-block:: genshi
 
@@ -212,21 +217,38 @@
     <?python
         from genshi.builder import tag
         def greeting(name):
-            return tag.b('Hello, %s!' % name') ?>
+            return tag.b('Hello, %s!' % name) ?>
     ${greeting('world')}
   </div>
 
 This will produce the following output:
 
-.. code-block:: genshi
+.. code-block:: xml
 
   <div>
     <b>Hello, world!</b>
   </div>
 
+In text templates (although only those using the new syntax introduced in
+Genshi 0.5), code blocks use the special ``{% python %}`` directive:
+
+.. code-block:: genshitext
+
+  {% python
+      from genshi.builder import tag
+      def greeting(name):
+          return 'Hello, %s!' % name
+  %}
+  ${greeting('world')}
+
+This will produce the following output::
+
+  Hello, world!
+
+
 Code blocks can import modules, define classes and functions, and basically do
 anything you can do in normal Python code. What code blocks can *not* do is to
-produce content that is included directly in the generated page.
+produce content that is emitted directly tp the generated output.
 
 .. note:: Using the ``print`` statement will print to the standard output
           stream, just as it does for other Python code in your application.
@@ -248,7 +270,11 @@
 into a sandboxable template engine; there are sufficient ways to do harm even
 using plain expressions.
 
-.. note:: Code blocks are not currently supported in text templates.
+.. warning:: Unfortunately, code blocks are severely limited when running
+             under Python 2.3: For example, it is not possible to access
+             variables defined in outer scopes. If you plan to use code blocks
+             extensively, it is strongly recommended that you run Python 2.4
+             or later.
 
 
 .. _`error handling`:
@@ -256,13 +282,56 @@
 Error Handling
 ==============
 
-By default, Genshi allows you to access variables that are not defined, without
-raising a ``NameError`` exception as regular Python code would:
+By default, Genshi raises an ``UndefinedError`` if a template expression
+attempts to access a variable that is not defined:
 
 .. code-block:: pycon
 
   >>> from genshi.template import MarkupTemplate
   >>> tmpl = MarkupTemplate('<p>${doh}</p>')
+  >>> tmpl.generate().render('xhtml')
+  Traceback (most recent call last):
+    ...
+  UndefinedError: "doh" not defined
+
+You can change this behavior by setting the variable lookup mode to "lenient".
+In that case, accessing undefined variables returns an `Undefined` object,
+meaning that the expression does not fail immediately. See below for details.
+
+If you need to check whether a variable exists in the template context, use the
+defined_ or the value_of_ function described below. To check for existence of
+attributes on an object, or keys in a dictionary, use the ``hasattr()``,
+``getattr()`` or ``get()`` functions, or the ``in`` operator, just as you would
+in regular Python code:
+
+  >>> from genshi.template import MarkupTemplate
+  >>> tmpl = MarkupTemplate('<p>${defined("doh")}</p>')
+  >>> print tmpl.generate().render('xhtml')
+  <p>False</p>
+
+.. note:: Lenient error handling was the default in Genshi prior to version 0.5.
+          Strict mode was introduced in version 0.4, and became the default in
+          0.5. The reason for this change was that the lenient error handling
+          was masking actual errors in templates, thereby also making it harder
+          to debug some problems.
+
+
+.. _`lenient`:
+
+Lenient Mode
+------------
+
+If you instruct Genshi to use the lenient variable lookup mode, it allows you
+to access variables that are not defined, without raising an ``UndefinedError``.
+
+This mode can be chosen by passing the ``lookup='lenient'`` keyword argument to
+the template initializer, or by passing the ``variable_lookup='lenient'``
+keyword argument to the ``TemplateLoader`` initializer:
+
+.. code-block:: pycon
+
+  >>> from genshi.template import MarkupTemplate
+  >>> tmpl = MarkupTemplate('<p>${doh}</p>', lookup='lenient')
   >>> print tmpl.generate().render('xhtml')
   <p></p>
 
@@ -272,7 +341,7 @@
 .. code-block:: pycon
 
   >>> from genshi.template import MarkupTemplate
-  >>> tmpl = MarkupTemplate('<p>${doh.oops}</p>')
+  >>> tmpl = MarkupTemplate('<p>${doh.oops}</p>', lookup='lenient')
   >>> print tmpl.generate().render('xhtml')
   Traceback (most recent call last):
     ...
@@ -284,42 +353,14 @@
 .. code-block:: pycon
 
   >>> from genshi.template import MarkupTemplate
-  >>> tmpl = MarkupTemplate('<p>${type(doh) is not Undefined}</p>')
+  >>> tmpl = MarkupTemplate('<p>${type(doh) is not Undefined}</p>',
+  ...                       lookup='lenient')
   >>> print tmpl.generate().render('xhtml')
   <p>False</p>
 
 Alternatively, the built-in functions defined_ or value_of_ can be used in this
 case.
 
-Strict Mode
------------
-
-In addition to the default "lenient" error handling, Genshi lets you use a less
-forgiving mode if you prefer errors blowing up loudly instead of being ignored
-silently.
-
-This mode can be chosen by passing the ``lookup='strict'`` keyword argument to
-the template initializer, or by passing the ``variable_lookup='strict'`` keyword
-argument to the ``TemplateLoader`` initializer:
-
-.. code-block:: pycon
-
-  >>> from genshi.template import MarkupTemplate
-  >>> tmpl = MarkupTemplate('<p>${doh}</p>', lookup='strict')
-  >>> print tmpl.generate().render('xhtml')
-  Traceback (most recent call last):
-    ...
-  UndefinedError: "doh" not defined
-
-When using strict mode, any reference to an undefined variable, as well as
-trying to access an non-existing item or attribute of an object, will cause an
-``UndefinedError`` to be raised immediately.
-
-.. note:: While this mode is currently not the default, it may be promoted to
-          the default in future versions of Genshi. In general, the default
-          lenient error handling mode can be considered dangerous as it silently
-          ignores typos.
-
 Custom Modes
 ------------
 
--- a/doc/text-templates.txt
+++ b/doc/text-templates.txt
@@ -6,10 +6,7 @@
 
 In addition to the XML-based template language, Genshi provides a simple
 text-based template language, intended for basic plain text generation needs.
-The language is similar to Cheetah_ or Velocity_.
-
-.. _cheetah: http://cheetahtemplate.org/
-.. _velocity: http://jakarta.apache.org/velocity/
+The language is similar to the Django_ template language.
 
 This document describes the template language and will be most useful as
 reference to those developing Genshi text templates. Templates are text files of
@@ -20,6 +17,13 @@
 See `Genshi Templating Basics <templates.html>`_ for general information on
 embedding Python code in templates.
 
+.. note:: Actually, Genshi currently has two different syntaxes for text
+          templates languages: One implemented by the class ``OldTextTemplate``
+          and another implemented by ``NewTextTemplate``. This documentation
+          concentrates on the latter, which is planned to completely replace the
+          older syntax. The older syntax is briefly described under legacy_.
+
+.. _django: http://www.djangoproject.com/
 
 .. contents:: Contents
    :depth: 3
@@ -32,38 +36,35 @@
 Template Directives
 -------------------
 
-Directives are lines starting with a ``#`` character followed immediately by
-the directive name. They can affect how the template is rendered in a number of
-ways: Genshi provides directives for conditionals and looping, among others.
+Directives are template commands enclosed by ``{% ... %}`` characters. They can
+affect how the template is rendered in a number of ways: Genshi provides
+directives for conditionals and looping, among others.
 
-Directives must be on separate lines, and the ``#`` character must be be the
-first non-whitespace character on that line. Each directive must be “closed”
-using a ``#end`` marker. You can add after the ``#end`` marker, for example to
-document which directive is being closed, or even the expression associated with 
-that directive. Any text after ``#end`` (but on the same line) is  ignored, 
-and effectively treated as a comment.
+Each directive must be terminated using an ``{% end %}`` marker. You can add
+a string inside the ``{% end %}`` marker, for example to document which
+directive is being closed, or even the expression associated with  that
+directive. Any text after ``end`` inside the delimiters is  ignored,  and
+effectively treated as a comment.
 
-If you want to include a literal ``#`` in the output, you need to escape it
-by prepending a backslash character (``\``). Note that this is **not** required 
-if the ``#`` isn't immediately followed by a letter, or it isn't the first
-non-whitespace character on the line.
+If you want to include a literal delimiter in the output, you need to escape it
+by prepending a backslash character (``\``).
 
 
 Conditional Sections
 ====================
 
-.. _`#if`:
+.. _`if`:
 
-``#if``
----------
+``{% if %}``
+------------
 
 The content is only rendered if the expression evaluates to a truth value:
 
 .. code-block:: genshitext
 
-  #if foo
+  {% if foo %}
     ${bar}
-  #end
+  {% end %}
 
 Given the data ``foo=True`` and ``bar='Hello'`` in the template context, this
 would produce::
@@ -71,58 +72,46 @@
     Hello
 
 
-.. _`#choose`:
-.. _`#when`:
-.. _`#otherwise`:
-
-``#choose``
--------------
+.. _`choose`:
+.. _`when`:
+.. _`otherwise`:
 
-The ``#choose`` directive, in combination with the directives ``#when`` and
-``#otherwise`` provides advanced contional processing for rendering one of
-several alternatives. The first matching ``#when`` branch is rendered, or, if
-no ``#when`` branch matches, the ``#otherwise`` branch is be rendered.
+``{% choose %}``
+----------------
 
-If the ``#choose`` directive has no argument the nested ``#when`` directives
-will be tested for truth:
+The ``choose`` directive, in combination with the directives ``when`` and
+``otherwise``, provides advanced contional processing for rendering one of
+several alternatives. The first matching ``when`` branch is rendered, or, if
+no ``when`` branch matches, the ``otherwise`` branch is be rendered.
+
+If the ``choose`` directive has no argument the nested ``when`` directives will
+be tested for truth:
 
 .. code-block:: genshitext
 
   The answer is:
-  #choose
-    #when 0 == 1
-      0
-    #end
-    #when 1 == 1
-      1
-    #end
-    #otherwise
-      2
-    #end
-  #end
+  {% choose %}
+    {% when 0 == 1 %}0{% end %}
+    {% when 1 == 1 %}1{% end %}
+    {% otherwise %}2{% end %}
+  {% end %}
 
 This would produce the following output::
 
   The answer is:
-      1
+    1
 
-If the ``#choose`` does have an argument, the nested ``#when`` directives will
-be tested for equality to the parent ``#choose`` value:
+If the ``choose`` does have an argument, the nested ``when`` directives will
+be tested for equality to the parent ``choose`` value:
 
 .. code-block:: genshitext
 
   The answer is:
-  #choose 1
-    #when 0
-      0
-    #end
-    #when 1
-      1
-    #end
-    #otherwise
-      2
-    #end
-  #end
+  {% choose 1 %}\
+    {% when 0 %}0{% end %}\
+    {% when 1 %}1{% end %}\
+    {% otherwise %}2{% end %}\
+  {% end %}
 
 This would produce the following output::
 
@@ -133,19 +122,19 @@
 Looping
 =======
 
-.. _`#for`:
+.. _`for`:
 
-``#for``
-----------
+``{% for %}``
+-------------
 
 The content is repeated for every item in an iterable:
 
 .. code-block:: genshitext
 
   Your items:
-  #for item in items
+  {% for item in items %}\
     * ${item}
-  #end
+  {% end %}
 
 Given ``items=[1, 2, 3]`` in the context data, this would produce::
 
@@ -158,21 +147,21 @@
 Snippet Reuse
 =============
 
-.. _`#def`:
+.. _`def`:
 .. _`macros`:
 
-``#def``
-----------
+``{% def %}``
+-------------
 
-The ``#def`` directive can be used to create macros, i.e. snippets of template
+The ``def`` directive can be used to create macros, i.e. snippets of template
 text that have a name and optionally some parameters, and that can be inserted
 in other places:
 
 .. code-block:: genshitext
 
-  #def greeting(name)
+  {% def greeting(name) %}
     Hello, ${name}!
-  #end
+  {% end %}
   ${greeting('world')}
   ${greeting('everyone else')}
 
@@ -181,15 +170,15 @@
     Hello, world!
     Hello, everyone else!
 
-If a macro doesn't require parameters, it can be defined as well as called
-without the parenthesis. For example:
+If a macro doesn't require parameters, it can be defined without the
+parenthesis. For example:
 
 .. code-block:: genshitext
 
-  #def greeting
+  {% def greeting %}
     Hello, world!
-  #end
-  ${greeting}
+  {% end %}
+  ${greeting()}
 
 The above would be rendered to::
 
@@ -197,17 +186,17 @@
 
 
 .. _includes:
-.. _`#include`:
+.. _`include`:
 
-``#include``
-------------
+``{% include %}``
+-----------------
 
 To reuse common parts of template text across template files, you can include
-other files using the ``#include`` directive:
+other files using the ``include`` directive:
 
 .. code-block:: genshitext
 
-  #include "base.txt"
+  {% include base.txt %}
 
 Any content included this way is inserted into the generated output. The
 included template sees the context data as it exists at the point of the
@@ -220,13 +209,13 @@
 the included file at "``myapp/base.txt``". You can also use Unix-style
 relative paths, for example "``../base.txt``" to look in the parent directory.
 
-Just like other directives, the argument to the ``#include`` directive accepts
+Just like other directives, the argument to the ``include`` directive accepts
 any Python expression, so the path to the included template can be determined
 dynamically:
 
 .. code-block:: genshitext
 
-  #include '%s.txt' % filename
+  {% include ${'%s.txt' % filename} %}
 
 Note that a ``TemplateNotFound`` exception is raised if an included file can't
 be found.
@@ -237,12 +226,12 @@
 Variable Binding
 ================
 
-.. _`#with`:
+.. _`with`:
 
-``#with``
------------
+``{% with %}``
+--------------
 
-The ``#with`` directive lets you assign expressions to variables, which can
+The ``{% with %}`` directive lets you assign expressions to variables, which can
 be used to make expressions inside the directive less verbose and more
 efficient. For example, if you need use the expression ``author.posts`` more
 than once, and that actually results in a database query, assigning the results
@@ -253,9 +242,9 @@
 .. code-block:: genshitext
 
   Magic numbers!
-  #with y=7; z=x+10
+  {% with y=7; z=x+10 %}
     $x $y $z
-  #end
+  {% end %}
 
 Given ``x=42`` in the context data, this would produce::
 
@@ -263,17 +252,109 @@
     42 7 52
 
 Note that if a variable of the same name already existed outside of the scope
-of the ``#with`` directive, it will **not** be overwritten. Instead, it will
-have the same value it had prior to the ``#with`` assignment. Effectively,
+of the ``with`` directive, it will **not** be overwritten. Instead, it will
+have the same value it had prior to the ``with`` assignment. Effectively,
 this means that variables are immutable in Genshi.
 
 
+.. _whitespace:
+
+---------------------------
+White-space and Line Breaks
+---------------------------
+
+Note that space or line breaks around directives is never automatically removed.
+Consider the following example:
+
+.. code-block:: genshitext
+
+  {% for item in items %}
+    {% if item.visible %}
+      ${item}
+    {% end %}
+  {% end %}
+
+This will result in two empty lines above and beneath every item, plus the
+spaces used for indentation. If you want to supress a line break, simply end
+the line with a backslash:
+
+.. code-block:: genshitext
+
+  {% for item in items %}\
+    {% if item.visible %}\
+      ${item}
+    {% end %}\
+  {% end %}\
+
+Now there would be no empty lines between the items in the output. But you still
+get the spaces used for indentation, and because the line breaks are removed,
+they actually continue and add up between lines. There are numerous ways to
+control white-space in the output while keeping the template readable, such as
+moving the indentation into the delimiters, or moving the end delimiter on the
+next line, and so on.
+
+
 .. _comments:
 
 --------
 Comments
 --------
 
-Lines where the first non-whitespace characters are ``##`` are removed from
-the output, and can thus be used for comments. This can be escaped using a
+Parts in templates can be commented out using the delimiters ``{# ... #}``.
+Any content in comments are removed from the output.
+
+.. code-block:: genshitext
+
+  {# This won't end up in the output #}
+  This will.
+
+Just like directive delimiters, these can be escaped by prefixing with a
 backslash.
+
+.. code-block:: genshitext
+
+  \{# This *will* end up in the output, including delimiters #}
+  This too.
+
+
+.. _legacy:
+
+---------------------------
+Legacy Text Template Syntax
+---------------------------
+
+The syntax for text templates was redesigned in version 0.5 of Genshi to make
+the language more flexible and powerful. The older syntax is based on line
+starting with dollar signs, similar to e.g. Cheetah_ or Velocity_.
+
+.. _cheetah: http://cheetahtemplate.org/
+.. _velocity: http://jakarta.apache.org/velocity/
+
+A simple template using the old syntax looked like this:
+
+.. code-block:: genshitext
+
+  Dear $name,
+  
+  We have the following items for you:
+  #for item in items
+   * $item
+  #end
+  
+  All the best,
+  Foobar
+
+Beyond the requirement of putting directives on separate lines prefixed with
+dollar signs, the language itself is very similar to the new one. Except that
+comments are lines that start with two ``#`` characters, and a line-break at the
+end of a directive is removed automatically.
+
+.. note:: If you're using this old syntax, it is strongly recommended to
+          migrate to the new syntax. Simply replace any references to
+          ``TextTemplate`` by ``NewTextTemplate`` (and also change the
+          text templates, of course). On the other hand, if you want to stick
+          with the old syntax for a while longer, replace references to
+          ``TextTemplate`` by ``OldTextTemplate``; while ``TextTemplate`` is
+          still an alias for the old language at this point, that will change
+          in a future release. But also note that the old syntax may be
+          dropped entirely in a future release.
new file mode 100644
--- /dev/null
+++ b/doc/upgrade.txt
@@ -0,0 +1,133 @@
+================
+Upgrading Genshi
+================
+
+
+.. contents:: Contents
+   :depth: 2
+.. sectnum::
+
+
+------------------------------------
+Upgrading from Genshi 0.4.x to 0.5.x
+------------------------------------
+
+Error Handling
+--------------
+
+The default error handling mode has been changed to "strict". This
+means that accessing variables not defined in the template data will
+now generate an immediate exception, as will accessing object
+attributes or dictionary keys that don't exist. If your templates rely
+on the old lenient behavior, you can configure Genshi to use that
+instead. See the documentation for details on how to do that. But be
+warned that lenient error handling may be removed completely in a
+future release.
+
+Match Template Processing
+-------------------------
+
+There has also been a subtle change to how ``py:match`` templates are
+processed: in previous versions, all match templates would be applied
+to the content generated by the matching template, and only the
+matching template itself was applied recursively to the original
+content. This behavior resulted in problems with many kinds of
+recursive matching, and hence was changed for 0.5: now, all match
+templates declared before the matching template are applied to the
+original content, and match templates declared after the matching
+template are applied to the generated content. This change should not
+have any effect on most applications, but you may want to check your
+use of match templates to make sure.
+
+Text Templates
+--------------
+
+Genshi 0.5 introduces a new, alternative syntax for text templates,
+which is more flexible and powerful compared to the old syntax. For
+backwards compatibility, this new syntax is not used by default,
+though it will be in a future version. It is recommended that you
+migrate to using this new syntax. To do so, simply rename any
+references in your code to ``TextTemplate`` to ``NewTextTemplate``. To
+explicitly use the old syntax, use ``OldTextTemplate`` instead, so
+that you can be sure you'll be using the same language when the
+default in Genshi is changed (at least until the old implementation is
+completely removed).
+
+``Markup`` Constructor
+----------------------
+
+The ``Markup`` class now longer has a specialized constructor. The old
+(undocumented) constructor provided a shorthand for doing positional
+substitutions. If you have code like this:
+
+.. code-block:: python
+
+  Markup('<b>%s</b>', name)
+
+You can simply replace it by the more explicit:
+
+.. code-block:: python
+
+  Markup('<b>%s</b>') % name
+
+``Template`` Constructor
+------------------------
+
+The constructor of the ``Template`` class and its subclasses has changed
+slightly: instead of the optional ``basedir`` parameter, it now expects
+an (also optional) ``filepath`` parameter, which specifies the absolute
+path to the template. You probably aren't using those constructors
+directly, anyway, but using the ``TemplateLoader`` API instead.
+
+
+------------------------------------
+Upgrading from Genshi 0.3.x to 0.4.x
+------------------------------------
+
+The modules ``genshi.filters`` and ``genshi.template`` have been
+refactored into packages containing multiple modules. While code using
+the regular APIs should continue to work without problems, you should
+make sure to remove any leftover traces of the files ``filters.py``
+and ``template.py`` in the ``genshi`` package on the installation
+path (including the corresponding ``.pyc`` files). This is not
+necessary when Genshi was installed as a Python egg.
+
+Results of evaluating template expressions are no longer implicitly
+called if they are callable. If you have been using that feature, you
+will need to add the parenthesis to actually call the function.
+
+Instances of ``genshi.core.Attrs`` are now immutable. Filters
+manipulating the attributes in a stream may need to be updated. Also,
+the ``Attrs`` class no longer automatically wraps all attribute names
+in ``QName`` objects, so users of the ``Attrs`` class need to do this
+themselves. See the documentation of the ``Attrs`` class for more
+information.
+
+
+---------------------
+Upgrading from Markup
+---------------------
+
+Prior to version 0.3, the name of the Genshi project was "Markup". The
+name change means that you will have to adjust your import statements
+and the namespace URI of XML templates, among other things:
+
+* The package name was changed from "markup" to "genshi". Please
+  adjust any import statements referring to the old package name.
+* The namespace URI for directives in Genshi XML templates has changed
+  from ``http://markup.edgewall.org/`` to
+  ``http://genshi.edgewall.org/``. Please update the ``xmlns:py``
+  declaration in your template files accordingly.
+
+Furthermore, due to the inclusion of a text-based template language,
+the class::
+
+  markup.template.Template
+
+has been renamed to::
+
+  genshi.template.MarkupTemplate
+
+If you've been using the Template class directly, you'll need to
+update your code (a simple find/replace should do—the API itself
+did not change).
--- a/doc/xml-templates.txt
+++ b/doc/xml-templates.txt
@@ -112,7 +112,7 @@
 Given the data ``foo=True`` and ``bar='Hello'`` in the template context, this
 would produce:
 
-.. code-block:: html
+.. code-block:: xml
 
   <div>
     <b>Hello</b>
@@ -153,7 +153,7 @@
 
 This would produce the following output:
 
-.. code-block:: html
+.. code-block:: xml
 
   <div>
     <span>1</span>
@@ -172,7 +172,7 @@
 
 This would produce the following output:
 
-.. code-block:: html
+.. code-block:: xml
 
   <div>
     <span>1</span>
@@ -197,7 +197,7 @@
 
 Given ``items=[1, 2, 3]`` in the context data, this would produce:
 
-.. code-block:: html
+.. code-block:: xml
 
   <ul>
     <li>1</li><li>2</li><li>3</li>
@@ -239,7 +239,7 @@
 
 The above would be rendered to:
 
-.. code-block:: html
+.. code-block:: xml
 
   <div>
     <p class="greeting">
@@ -264,7 +264,7 @@
 
 The above would be rendered to:
 
-.. code-block:: html
+.. code-block:: xml
 
   <div>
     <p class="greeting">
@@ -307,7 +307,7 @@
 
 This would result in the following output:
 
-.. code-block:: html
+.. code-block:: xml
 
   <div>
     <span>
@@ -322,6 +322,12 @@
 
 .. _`Using XPath`: streams.html#using-xpath
 
+Match templates are applied both to the original markup as well to the
+generated markup. The order in which they are applied depends on the order
+they are declared in the template source: a match template defined after
+another match template is applied to the output generated by the first match
+template. The match templates basically form a pipeline.
+
 This directive can also be used as an element:
 
 .. code-block:: genshi
@@ -333,6 +339,54 @@
     <greeting name="Dude" />
   </div>
 
+When used this way, the ``py:match`` directive can also be annotated with a
+couple of optimization hints. For example, the following informs the matching
+engine that the match should only be applied once:
+
+.. code-block:: genshi
+
+  <py:match path="body" once="true">
+    <body py:attrs="select('@*')">
+      <div id="header">...</div>
+      ${select("*|text()")}
+      <div id="footer">...</div>
+    </body>
+  </py:match>
+
+The following optimization hints are recognized:
+
++---------------+-----------+-----------------------------------------------+
+| Attribute     | Default   | Description                                   |
++===============+===========+===============================================+
+| ``buffer``    | ``true``  | Whether the matched content should be         |
+|               |           | buffered in memory. Buffering can improve     |
+|               |           | performance a bit at the cost of needing more |
+|               |           | memory during rendering. Buffering is         |
+|               |           | ''required'' for match templates that contain |
+|               |           | more than one invocation of the ``select()``  |
+|               |           | function. If there is only one call, and the  |
+|               |           | matched content can potentially be very long, |
+|               |           | consider disabling buffering to avoid         |
+|               |           | excessive memory use.                         |
++---------------+-----------+-----------------------------------------------+
+| ``once``      | ``false`` | Whether the engine should stop looking for    |
+|               |           | more matching elements after the first match. |
+|               |           | Use this on match templates that match        |
+|               |           | elements that can only occur once in the      |
+|               |           | stream, such as the ``<head>`` or ``<body>``  |
+|               |           | elements in an HTML template, or elements     |
+|               |           | with a specific ID.                           |
++---------------+-----------+-----------------------------------------------+
+| ``recursive`` | ``true``  | Whether the match template should be applied  |
+|               |           | to its own output. Note that ``once`` implies |
+|               |           | non-recursive behavior, so this attribute     |
+|               |           | only needs to be set for match templates that |
+|               |           | don't also have ``once`` set.                 |
++---------------+-----------+-----------------------------------------------+
+
+.. note:: The ``py:match`` optimization hints were added in the 0.5 release. In
+          earlier versions, the attributes have no effect.
+
 
 Variable Binding
 ================
@@ -358,7 +412,7 @@
 
 Given ``x=42`` in the context data, this would produce:
 
-.. code-block:: html
+.. code-block:: xml
 
   <div>
     <span>42 7 52</span>
@@ -397,7 +451,7 @@
 Given ``foo={'class': 'collapse'}`` in the template context, this would
 produce:
 
-.. code-block:: html
+.. code-block:: xml
 
   <ul>
     <li class="collapse">Bar</li>
@@ -406,7 +460,7 @@
 Attributes with the value ``None`` are omitted, so given ``foo={'class': None}``
 in the context for the same template this would produce:
 
-.. code-block:: html
+.. code-block:: xml
 
   <ul>
     <li>Bar</li>
@@ -431,7 +485,7 @@
 
 Given ``bar='Bye'`` in the context data, this would produce:
 
-.. code-block:: html
+.. code-block:: xml
 
   <ul>
     <li>Bye</li>
@@ -456,13 +510,20 @@
 
 Given ``bar='Bye'`` in the context data, this would produce:
 
-.. code-block:: html
+.. code-block:: xml
 
   <div>
     Bye
   </div>
 
-This directive can only be used as an attribute.
+This directive can also be used as an element (since version 0.5):
+
+.. code-block:: genshi
+
+  <div>
+    <py:replace value="title">Placeholder</py:replace>
+  </div>
+
 
 
 .. _`py:strip`:
@@ -482,7 +543,7 @@
 
 This would be rendered as:
 
-.. code-block:: html
+.. code-block:: xml
 
   <div>
     <b>foo</b>
@@ -564,6 +625,10 @@
 
 .. _`xinclude specification`: http://www.w3.org/TR/xinclude/
 
+
+Dynamic Includes
+================
+
 Incudes in Genshi are fully dynamic: Just like normal attributes, the `href`
 attribute accepts expressions, and directives_ can be used on the
 ``<xi:include />`` element just as on any other element, meaning you can do
@@ -575,6 +640,23 @@
               py:for="name in ('foo', 'bar', 'baz')" />
 
 
+Including Text Templates
+========================
+
+The ``parse`` attribute of the ``<xi:include>`` element can be used to specify
+whether the included template is an XML template or a text template (using the
+new syntax added in Genshi 0.5):
+
+.. code-block:: genshi
+
+  <xi:include href="myscript.js" parse="text" />
+
+This example would load the ``myscript.js`` file as a ``NewTextTemplate``. See
+`text templates`_ for details on the syntax of text templates.
+
+.. _`text templates`: text-templates.html
+
+
 .. _comments:
 
 --------
--- a/examples/bench/basic.py
+++ b/examples/bench/basic.py
@@ -9,7 +9,8 @@
 import sys
 import timeit
 
-__all__ = ['clearsilver', 'mako', 'django', 'kid', 'genshi', 'simpletal']
+__all__ = ['clearsilver', 'mako', 'django', 'kid', 'genshi', 'genshi_text',
+           'simpletal']
 
 def genshi(dirname, verbose=False):
     from genshi.template import TemplateLoader
@@ -24,6 +25,20 @@
         print render()
     return render
 
+def genshi_text(dirname, verbose=False):
+    from genshi.core import escape
+    from genshi.template import TemplateLoader, NewTextTemplate
+    loader = TemplateLoader([dirname], auto_reload=False)
+    template = loader.load('template.txt', cls=NewTextTemplate)
+    def render():
+        data = dict(escape=escape, title='Just a test', user='joe',
+                    items=['Number %d' % num for num in range(1, 15)])
+        return template.generate(**data).render('text')
+
+    if verbose:
+        print render()
+    return render
+
 def mako(dirname, verbose=False):
     from mako.lookup import TemplateLookup
     lookup = TemplateLookup(directories=[dirname], filesystem_checks=False)
@@ -107,7 +122,7 @@
     try:
         import kid
     except ImportError:
-        print>>sys.stderr, "SimpleTAL not installed, skipping"
+        print>>sys.stderr, "Kid not installed, skipping"
         return lambda: None
     kid.path = kid.TemplatePath([dirname])
     template = kid.load_template('template.kid').Template
--- a/examples/bench/bigtable.py
+++ b/examples/bench/bigtable.py
@@ -10,7 +10,7 @@
 import timeit
 from StringIO import StringIO
 from genshi.builder import tag
-from genshi.template import MarkupTemplate
+from genshi.template import MarkupTemplate, NewTextTemplate
 
 try:
     from elementtree import ElementTree as et
@@ -60,6 +60,14 @@
 <table xmlns:py="http://genshi.edgewall.org/">$table</table>
 """)
 
+genshi_text_tmpl = NewTextTemplate("""
+<table>
+{% for row in table %}<tr>
+{% for c in row.values() %}<td>$c</td>{% end %}
+</tr>{% end %}
+</table>
+""")
+
 if DjangoTemplate:
     django_tmpl = DjangoTemplate("""
     <table>
@@ -95,6 +103,11 @@
     stream = genshi_tmpl.generate(table=table)
     stream.render('html', strip_whitespace=False)
 
+def test_genshi_text():
+    """Genshi text template"""
+    stream = genshi_text_tmpl.generate(table=table)
+    stream.render('text')
+
 def test_genshi_builder():
     """Genshi template + tag builder"""
     stream = tag.TABLE([
@@ -183,9 +196,9 @@
 
 
 def run(which=None, number=10):
-    tests = ['test_builder', 'test_genshi', 'test_genshi_builder',
-             'test_mako', 'test_kid', 'test_kid_et', 'test_et', 'test_cet',
-             'test_clearsilver', 'test_django']
+    tests = ['test_builder', 'test_genshi', 'test_genshi_text',
+             'test_genshi_builder', 'test_mako', 'test_kid', 'test_kid_et',
+             'test_et', 'test_cet', 'test_clearsilver', 'test_django']
 
     if which:
         tests = filter(lambda n: n[5:] in which, tests)
--- a/examples/bench/genshi/base.html
+++ b/examples/bench/genshi/base.html
@@ -6,12 +6,12 @@
     Hello, ${name}!
   </p>
 
-  <body py:match="body">
+  <py:match path="body" once="true"><body>
     <div id="header">
       <h1>${title}</h1>
     </div>
     ${select('*')}
     <div id="footer" />
-  </body>
+  </body></py:match>
 
 </html>
new file mode 100644
--- /dev/null
+++ b/examples/bench/genshi_text/footer.txt
@@ -0,0 +1,1 @@
+<div id="footer"></div>
new file mode 100644
--- /dev/null
+++ b/examples/bench/genshi_text/header.txt
@@ -0,0 +1,3 @@
+<div id="header">
+  <h1>${escape(title)}</h1>
+</div>
new file mode 100644
--- /dev/null
+++ b/examples/bench/genshi_text/template.txt
@@ -0,0 +1,28 @@
+<!DOCTYPE html
+    PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
+    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" lang="en">
+
+<head>
+  <title>${escape(title)}</title>
+</head>
+
+<body>
+  {% include header.txt %}
+  {% def greeting(name) %}
+    Hello, ${name}!
+  {% end %}
+
+  <div>${greeting(user)}</div>
+  <div>${greeting("me")}</div>
+  <div>${greeting("world")}</div>
+  
+  <h2>Loop</h2>
+  {% if items %}<ul>
+    {% for idx, item in enumerate(items) %}\
+      <li{% if idx + 1 == len(items) %} class="last"{% end %}>${escape(item)}</li>
+    {% end %}
+  </ul>{% end %}
+  
+  {% include footer.txt %}
+</body>
deleted file mode 100644
--- a/examples/cherrypy/config.txt
+++ /dev/null
@@ -1,3 +0,0 @@
-[global]
-server.socket_port = 8000
-server.thread_pool = 10
deleted file mode 100644
--- a/examples/cherrypy/index.html
+++ /dev/null
@@ -1,11 +0,0 @@
-<!DOCTYPE html
-    PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
-    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
-<html xmlns="http://www.w3.org/1999/xhtml"
-      xmlns:py="http://genshi.edgewall.org/"
-      xmlns:xi="http://www.w3.org/2001/XInclude"
-      lang="en">
-  <body>
-    <span class="greeting">Hello, ${name}!</span>
-  </body>
-</html>
deleted file mode 100644
--- a/examples/cherrypy/index.py
+++ /dev/null
@@ -1,19 +0,0 @@
-import os
-import sys
-
-import cherrypy
-from genshi.template import TemplateLoader
-
-loader = TemplateLoader([os.path.dirname(os.path.abspath(__file__))])
-
-
-class Example(object):
-
-    @cherrypy.expose
-    def index(self):
-        tmpl = loader.load('index.html')
-        return tmpl.generate(name='world').render('xhtml')
-
-
-if __name__ == '__main__':
-    cherrypy.quickstart(Example(), config='config.txt')
deleted file mode 100644
--- a/examples/includes/common/macros.html
+++ /dev/null
@@ -1,12 +0,0 @@
-<div xmlns:py="http://genshi.edgewall.org/"
-     py:strip="">
-  <div py:def="macro1">reference me, please</div>
-  <div py:def="macro2(name, classname='expanded')" class="${classname}">
-   Hello ${name.title()}
-  </div>
-  <span py:match="greeting" class="greeting">
-    Hello ${select('@name')}
-  </span>
-  <span py:match="span[@class='greeting']" style="text-decoration: underline" 
-        py:content="select('text()')"/>
-</div>
deleted file mode 100644
--- a/examples/includes/module/test.html
+++ /dev/null
@@ -1,22 +0,0 @@
-<!DOCTYPE html
-    PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
-    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
-<html xmlns="http://www.w3.org/1999/xhtml"
-      xmlns:py="http://genshi.edgewall.org/"
-      xmlns:xi="http://www.w3.org/2001/XInclude">
- <xi:include href="${skin}/layout.html" />
- <xi:include href="custom_stuff.html"><xi:fallback/></xi:include>
- <body class="$bozz">
-  <ul py:attrs="{'id': 'second', 'class': None}" py:if="len(items) > 0">
-   <li py:for="item in items">Item ${item.split()[-1]}</li>
-   XYZ ${hey}
-  </ul>
-  ${macro1()} ${macro1()} ${macro1()}
-  ${macro2('john')}
-  ${macro2('kate', classname='collapsed')}
-  <div py:content="macro2('helmut')" py:strip="">Replace me</div>
-  <greeting name="Dude" />
-  <greeting name="King" />
-  <span class="greeting">Hello Silicon</span>
- </body>
-</html>
deleted file mode 100755
--- a/examples/includes/run.py
+++ /dev/null
@@ -1,49 +0,0 @@
-#!/usr/bin/python
-# -*- coding: utf-8 -*-
-
-import os
-import sys
-import timing
-
-from genshi.template import Context, TemplateLoader
-
-def test():
-    base_path = os.path.dirname(os.path.abspath(__file__))
-    loader = TemplateLoader([os.path.join(base_path, 'skins'),
-                             os.path.join(base_path, 'module'),
-                             os.path.join(base_path, 'common')])
-
-    timing.start()
-    tmpl = loader.load('test.html')
-    timing.finish()
-    print ' --> parse stage: %dms' % timing.milli()
-
-    data = dict(hello='<world>', skin='default', hey='ZYX', bozz=None,
-                items=['Number %d' % num for num in range(1, 15)])
-
-    print tmpl.generate(Context(**data)).render(method='html')
-
-    times = []
-    for i in range(100):
-        timing.start()
-        list(tmpl.generate(Context(**data)))
-        timing.finish()
-        times.append(timing.milli())
-        sys.stdout.write('.')
-        sys.stdout.flush()
-    print
-
-    print ' --> render stage: %dms (avg), %dms (min), %dms (max)' % (
-          sum(times) / len(times), min(times), max(times))
-
-if __name__ == '__main__':
-    if '-p' in sys.argv:
-        import hotshot, hotshot.stats
-        prof = hotshot.Profile("template.prof")
-        benchtime = prof.runcall(test)
-        stats = hotshot.stats.load("template.prof")
-        stats.strip_dirs()
-        stats.sort_stats('time', 'calls')
-        stats.print_stats()
-    else:
-        test()
deleted file mode 100644
--- a/examples/includes/skins/default/footer.html
+++ /dev/null
@@ -1,4 +0,0 @@
-<div id="footer">
-  <hr />
-  <h1>And goodbye</h1>
-</div>
deleted file mode 100644
--- a/examples/includes/skins/default/header.html
+++ /dev/null
@@ -1,3 +0,0 @@
-<div id="masthead">
-  <h1>Welcome</h1>
-</div>
\ No newline at end of file
deleted file mode 100644
--- a/examples/includes/skins/default/layout.html
+++ /dev/null
@@ -1,17 +0,0 @@
-<html xmlns="http://www.w3.org/1999/xhtml"
-      xmlns:py="http://genshi.edgewall.org/"
-      xmlns:xi="http://www.w3.org/2001/XInclude"
-      py:strip="">
- <xi:include href="../macros.html" />
- <head>
-  <title>Hello ${hello}</title>
-  <style type="text/css">@import(style.css)</style>
- </head>
- <body py:match="body">
-   <xi:include href="header.html" />
-   <div id="content">
-     ${select('*')}
-   </div>
-   <xi:include href="footer.html" />
- </body>
-</html>
deleted file mode 100644
--- a/examples/transform/README.txt
+++ /dev/null
@@ -1,3 +0,0 @@
-This example shows how to transform some HTML input, while adding
-elements such as headers, and using <em>/<strong> instead of <i>/<b>.
-The output is that a proper XTHML document.
deleted file mode 100644
--- a/examples/transform/index.html
+++ /dev/null
@@ -1,24 +0,0 @@
-<HTML>
- <HEAD>
-  <TITLE>Aaarrgh</TITLE>
-  <LINK REL=stylesheet href='badstyle.css'>
- </HEAD>
- 
- <BODY>
-  <H1>Aaargh</H1>
-  <P>
-    <B>Lorem <I>ipsum</I></B> dolor sit amet, consectetur<BR>
-    adipisicing elit, sed do eiusmod tempor incididunt ut<BR>
-    labore et dolore magna aliqua. Ut enim ad minim veniam,<BR>
-    quis nostrud exercitation ullamco laboris nisi ut<BR>
-    aliquip ex ea commodo consequat.
-  </P>
-  <P>
-    Duis aute irure dolor in reprehenderit in voluptate velit<BR>
-    esse cillum dolore eu fugiat nulla pariatur. Excepteur sint<BR>
-    occaecat cupidatat non proident, sunt in culpa qui officia<BR>
-    deserunt mollit anim <I>id est laborum</I>.
-  </P>
- </BODY>
-
-</HTML>
deleted file mode 100644
--- a/examples/transform/run.py
+++ /dev/null
@@ -1,24 +0,0 @@
-#!/usr/bin/python
-# -*- coding: utf-8 -*-
-
-import os
-import sys
-
-from genshi.input import HTMLParser
-from genshi.template import Context, MarkupTemplate
-
-def transform(html_filename, tmpl_filename):
-    tmpl_fileobj = open(tmpl_filename)
-    tmpl = MarkupTemplate(tmpl_fileobj, tmpl_filename)
-    tmpl_fileobj.close()
-
-    html_fileobj = open(html_filename)
-    html = HTMLParser(html_fileobj, html_filename)
-    print tmpl.generate(Context(input=html)).render('xhtml')
-    html_fileobj.close()
-
-if __name__ == '__main__':
-    basepath = os.path.dirname(os.path.abspath(__file__))
-    tmpl_filename = os.path.join(basepath, 'template.xml')
-    html_filename = os.path.join(basepath, 'index.html')
-    transform(html_filename, tmpl_filename)
deleted file mode 100644
--- a/examples/transform/template.xml
+++ /dev/null
@@ -1,22 +0,0 @@
-<!DOCTYPE html
-    PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
-    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
-<html xmlns:py="http://genshi.edgewall.org/" py:strip="">
-
-  <!--! Add a header DIV on top of every page with a logo image -->
-  <body py:match="BODY|body">
-    <div id="header">
-      <img src="logo.png" alt="Bad Style"/>
-    </div>
-    ${select('*')}
-  </body>
-
-  <!--! Use semantic instead of presentational tags for emphasis -->
-  <strong py:match="B|b">${select('*|text()')}</strong>
-  <em py:match="I|i">${select('*|text()')}</em>
-
-  <!--! Include the actual HTML stream, which will be processed by the rules
-        defined above -->
-  ${input}
-
-</html>
new file mode 100644
new file mode 100755
--- /dev/null
+++ b/examples/tutorial/geddit/controller.py
@@ -0,0 +1,138 @@
+#!/usr/bin/env python
+
+import operator, os, pickle, sys
+
+import cherrypy
+from formencode import Invalid
+from genshi.input import HTML
+from genshi.filters import HTMLFormFiller, HTMLSanitizer
+
+from geddit.form import LinkForm, CommentForm
+from geddit.lib import ajax, template
+from geddit.model import Link, Comment
+
+
+class Root(object):
+
+    def __init__(self, data):
+        self.data = data
+
+    @cherrypy.expose
+    @template.output('index.html')
+    def index(self):
+        links = sorted(self.data.values(), key=operator.attrgetter('time'))
+        return template.render(links=links)
+
+    @cherrypy.expose
+    @template.output('submit.html')
+    def submit(self, cancel=False, **data):
+        if cherrypy.request.method == 'POST':
+            if cancel:
+                raise cherrypy.HTTPRedirect('/')
+            form = LinkForm()
+            try:
+                data = form.to_python(data)
+                link = Link(**data)
+                self.data[link.id] = link
+                raise cherrypy.HTTPRedirect('/')
+            except Invalid, e:
+                errors = e.unpack_errors()
+        else:
+            errors = {}
+
+        return template.render(errors=errors) | HTMLFormFiller(data=data)
+
+    @cherrypy.expose
+    @template.output('info.html')
+    def info(self, id):
+        link = self.data.get(id)
+        if not link:
+            raise cherrypy.NotFound()
+        return template.render(link=link)
+
+    @cherrypy.expose
+    @template.output('comment.html')
+    def comment(self, id, cancel=False, **data):
+        link = self.data.get(id)
+        if not link:
+            raise cherrypy.NotFound()
+        if cherrypy.request.method == 'POST':
+            if cancel:
+                raise cherrypy.HTTPRedirect('/info/%s' % link.id)
+            form = CommentForm()
+            try:
+                data = form.to_python(data)
+                markup = HTML(data['content']) | HTMLSanitizer()
+                data['content'] = markup.render('xhtml')
+                comment = link.add_comment(**data)
+                if not ajax.is_xhr():
+                    raise cherrypy.HTTPRedirect('/info/%s' % link.id)
+                return template.render('_comment.html', comment=comment,
+                                       num=len(link.comments))
+            except Invalid, e:
+                errors = e.unpack_errors()
+        else:
+            errors = {}
+
+        if ajax.is_xhr():
+            stream = template.render('_form.html', link=link, errors=errors)
+        else:
+            stream = template.render(link=link, comment=None, errors=errors)
+        return stream | HTMLFormFiller(data=data)
+
+    @cherrypy.expose
+    @template.output('index.xml', method='xml')
+    def feed(self, id=None):
+        if id:
+            link = self.data.get(id)
+            if not link:
+                raise cherrypy.NotFound()
+            return template.render('info.xml', link=link)
+        else:
+            links = sorted(self.data.values(), key=operator.attrgetter('time'))
+            return template.render(links=links)
+
+
+def main(filename):
+    # load data from the pickle file, or initialize it to an empty list
+    if os.path.exists(filename):
+        fileobj = open(filename, 'rb')
+        try:
+            data = pickle.load(fileobj)
+        finally:
+            fileobj.close()
+    else:
+        data = {}
+
+    def _save_data():
+        # save data back to the pickle file
+        fileobj = open(filename, 'wb')
+        try:
+            pickle.dump(data, fileobj)
+        finally:
+            fileobj.close()
+    if hasattr(cherrypy.engine, 'subscribe'): # CherryPy >= 3.1
+        cherrypy.engine.subscribe('stop', _save_data)
+    else:
+        cherrypy.engine.on_stop_engine_list.append(_save_data)
+
+    # Some global configuration; note that this could be moved into a
+    # configuration file
+    cherrypy.config.update({
+        'tools.encode.on': True, 'tools.encode.encoding': 'utf-8',
+        'tools.decode.on': True,
+        'tools.trailing_slash.on': True,
+        'tools.staticdir.root': os.path.abspath(os.path.dirname(__file__)),
+    })
+
+    cherrypy.quickstart(Root(data), '/', {
+        '/media': {
+            'tools.staticdir.on': True,
+            'tools.staticdir.dir': 'static'
+        }
+    })
+
+if __name__ == '__main__':
+    import formencode
+    formencode.api.set_stdtranslation(languages=['en'])
+    main(sys.argv[1])
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/form.py
@@ -0,0 +1,12 @@
+from formencode import Schema, validators
+
+
+class LinkForm(Schema):
+    username = validators.UnicodeString(not_empty=True)
+    url = validators.URL(not_empty=True, add_http=True, check_exists=False)
+    title = validators.UnicodeString(not_empty=True)
+
+
+class CommentForm(Schema):
+    username = validators.UnicodeString(not_empty=True)
+    content = validators.UnicodeString(not_empty=True)
new file mode 100644
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/lib/ajax.py
@@ -0,0 +1,5 @@
+import cherrypy
+
+def is_xhr():
+    requested_with = cherrypy.request.headers.get('X-Requested-With')
+    return requested_with and requested_with.lower() == 'xmlhttprequest'
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/lib/template.py
@@ -0,0 +1,47 @@
+import os
+
+import cherrypy
+from genshi.core import Stream
+from genshi.output import encode, get_serializer
+from genshi.template import Context, TemplateLoader
+
+from geddit.lib import ajax
+
+loader = TemplateLoader(
+    os.path.join(os.path.dirname(__file__), '..', 'templates'),
+    auto_reload=True
+)
+
+def output(filename, method='html', encoding='utf-8', **options):
+    """Decorator for exposed methods to specify what template the should use
+    for rendering, and which serialization method and options should be
+    applied.
+    """
+    def decorate(func):
+        def wrapper(*args, **kwargs):
+            cherrypy.thread_data.template = loader.load(filename)
+            opt = options.copy()
+            if not ajax.is_xhr() and method == 'html':
+                opt.setdefault('doctype', 'html')
+            serializer = get_serializer(method, **opt)
+            stream = func(*args, **kwargs)
+            if not isinstance(stream, Stream):
+                return stream
+            return encode(serializer(stream), method=serializer,
+                          encoding=encoding)
+        return wrapper
+    return decorate
+
+def render(*args, **kwargs):
+    """Function to render the given data to the template specified via the
+    ``@output`` decorator.
+    """
+    if args:
+        assert len(args) == 1, \
+            'Expected exactly one argument, but got %r' % (args,)
+        template = loader.load(args[0])
+    else:
+        template = cherrypy.thread_data.template
+    ctxt = Context(url=cherrypy.url)
+    ctxt.push(kwargs)
+    return template.generate(ctxt)
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/model.py
@@ -0,0 +1,31 @@
+from datetime import datetime
+
+
+class Link(object):
+
+    def __init__(self, username, url, title):
+        self.username = username
+        self.url = url
+        self.title = title
+        self.time = datetime.utcnow()
+        self.id = hex(hash(tuple([username, url, title, self.time])))[2:]
+        self.comments = []
+
+    def __repr__(self):
+        return '<%s %r>' % (type(self).__name__, self.title)
+
+    def add_comment(self, username, content):
+        comment = Comment(username, content)
+        self.comments.append(comment)
+        return comment
+
+
+class Comment(object):
+
+    def __init__(self, username, content):
+        self.username = username
+        self.content = content
+        self.time = datetime.utcnow()
+
+    def __repr__(self):
+        return '<%s by %r>' % (type(self).__name__, self.username)
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/static/jquery.js
@@ -0,0 +1,11 @@
+/*
+ * jQuery 1.1.4 - New Wave Javascript
+ *
+ * Copyright (c) 2007 John Resig (jquery.com)
+ * Dual licensed under the MIT (MIT-LICENSE.txt)
+ * and GPL (GPL-LICENSE.txt) licenses.
+ *
+ * $Date: 2007-08-23 21:49:27 -0400 (Thu, 23 Aug 2007) $
+ * $Rev: 2862 $
+ */
+eval(function(p,a,c,k,e,r){e=function(c){return(c<a?'':e(parseInt(c/a)))+((c=c%a)>35?String.fromCharCode(c+29):c.toString(36))};if(!''.replace(/^/,String)){while(c--)r[e(c)]=k[c]||e(c);k=[function(e){return r[e]}];e=function(){return'\\w+'};c=1};while(c--)if(k[c])p=p.replace(new RegExp('\\b'+e(c)+'\\b','g'),k[c]);return p}('(9(){6(1f C!="Q")E v=C;E C=19.16=9(a,c){6(19==7||!7.4a)F 1s C(a,c);F 7.4a(a,c)};6(1f $!="Q")E B=$;19.$=C;E q=/^[^<]*(<(.|\\s)+>)[^>]*$|^#(\\w+)$/;C.15=C.3v={4a:9(a,c){a=a||R;6(1f a=="1E"){E m=q.2d(a);6(m&&(m[1]||!c)){6(m[1])a=C.3c([m[1]]);G{E b=R.37(m[3]);6(b)6(b.2j!=m[3])F C().1F(a);G{7[0]=b;7.H=1;F 7}G a=[]}}G F 1s C(c).1F(a)}G 6(C.1g(a))F 1s C(R)[C.15.1L?"1L":"2f"](a);F 7.5J(a.1b==1K&&a||(a.3w||a.H&&a!=19&&!a.1t&&a[0]!=Q&&a[0].1t)&&C.2V(a)||[a])},3w:"1.1.4",7K:9(){F 7.H},H:0,21:9(a){F a==Q?C.2V(7):7[a]},1O:9(a){E b=C(a);b.5c=7;F b},5J:9(a){7.H=0;1K.3v.Y.T(7,a);F 7},J:9(a,b){F C.J(7,a,b)},45:9(a){E b=-1;7.J(9(i){6(7==a)b=i});F b},1j:9(f,d,e){E c=f;6(f.1b==3n)6(d==Q)F 7.H&&C[e||"1j"](7[0],f)||Q;G{c={};c[f]=d}F 7.J(9(a){I(E b 17 c)C.1j(e?7.S:7,b,C.4Q(7,c[b],e,a,b))})},1h:9(b,a){F 7.1j(b,a,"34")},2Q:9(e){6(1f e!="4P"&&e!=K)F 7.3K().3H(R.60(e));E t="";C.J(e||7,9(){C.J(7.2Z,9(){6(7.1t!=8)t+=7.1t!=1?7.5S:C.15.2Q([7])})});F t},82:9(){E a,2e=1a;F 7.J(9(){6(!a)a=C.3c(2e,7.2I);E b=a[0].3B(O);7.P.2p(b,7);20(b.1k)b=b.1k;b.4p(7)})},3H:9(){F 7.2J(1a,O,1,9(a){7.4p(a)})},5v:9(){F 7.2J(1a,O,-1,9(a){7.2p(a,7.1k)})},5u:9(){F 7.2J(1a,M,1,9(a){7.P.2p(a,7)})},5t:9(){F 7.2J(1a,M,-1,9(a){7.P.2p(a,7.2a)})},3L:9(){F 7.5c||C([])},1F:9(t){E b=C.3M(7,9(a){F C.1F(t,a)});F 7.1O(/[^+>] [^+>]/.1d(t)||t.U("..")>-1?C.4d(b):b)},7o:9(e){e=e!=Q?e:O;E d=7.1r(7.1F("*"));6(C.N.12){d.J(9(){7.2l$1i={};I(E a 17 7.$1i)7.2l$1i[a]=C.14({},7.$1i[a])}).49()}E r=7.1O(C.3M(7,9(a){F a.3B(e)}));6(C.N.12){d.J(9(){E c=7.2l$1i;I(E a 17 c)I(E b 17 c[a])C.1c.1r(7,a,c[a][b],c[a][b].V);7.2l$1i=K})}6(e){E f=r.1r(r.1F(\'*\')).1l(\'2b,39[@L=3i]\');d.1l(\'2b,39[@L=3i]\').J(9(i){6(7.3j)f[i].3j=7.3j;6(7.27)f[i].27=O})}F r},1l:9(t){F 7.1O(C.1g(t)&&C.2B(7,9(b,a){F t.T(b,[a])})||C.2R(t,7))},5l:9(t){F 7.1O(t.1b==3n&&C.2R(t,7,O)||C.2B(7,9(a){F(t.1b==1K||t.3w)?C.4K(a,t)<0:a!=t}))},1r:9(t){F 7.1O(C.29(7.21(),t.1b==3n?C(t).21():t.H!=Q&&(!t.W||t.W=="6s")?t:[t]))},3y:9(a){F a?C.2R(a,7).H>0:M},2G:9(a){F a==Q?(7.H?7[0].2A:K):7.1j("2A",a)},5W:9(a){F a==Q?(7.H?7[0].2W:K):7.3K().3H(a)},3S:9(){F 7.1O(1K.3v.3S.T(7,1a))},2J:9(f,d,g,e){E c=7.H>1,a;F 7.J(9(){6(!a){a=C.3c(f,7.2I);6(g<0)a.8E()}E b=7;6(d&&C.W(7,"1A")&&C.W(a[0],"3O"))b=7.4L("1w")[0]||7.4p(R.6a("1w"));C.J(a,9(){6(C.W(7,"33")){6(7.32)C.31({1G:7.32,2w:M,3G:"33"});G C.4E(7.2Q||7.5Z||7.2W||"")}G e.T(b,[c?7.3B(O):7])})})}};C.14=C.15.14=9(){E c=1a[0]||{},a=1,1M=1a.H,4D=M;6(c.1b==8d){4D=c;c=1a[1]||{}}6(1M==1){c=7;a=0}E b;I(;a<1M;a++)6((b=1a[a])!=K)I(E i 17 b){6(c==b[i])5X;6(4D&&1f b[i]==\'4P\'&&c[i])C.14(c[i],b[i]);G 6(b[i]!=Q)c[i]=b[i]}F c};C.14({8a:9(a){19.$=B;6(a)19.16=v;F C},1g:9(a){F!!a&&1f a!="1E"&&!a.W&&a.1b!=1K&&/9/i.1d(a+"")},3E:9(a){F a.3D&&!a.4z||a.4y&&a.2I&&!a.2I.4z},4E:9(a){a=C.2s(a);6(a){6(19.5N)19.5N(a);G 6(C.N.1H)19.4x(a,0);G 2T.2S(19,a)}},W:9(b,a){F b.W&&b.W.1I()==a.1I()},J:9(a,b,c){6(c){6(a.H==Q)I(E i 17 a)b.T(a[i],c);G I(E i=0,3A=a.H;i<3A;i++)6(b.T(a[i],c)===M)1J}G{6(a.H==Q)I(E i 17 a)b.2S(a[i],i,a[i]);G I(E i=0,3A=a.H,2G=a[0];i<3A&&b.2S(2G,i,2G)!==M;2G=a[++i]){}}F a},4Q:9(c,b,d,e,a){6(C.1g(b))b=b.2S(c,[e]);E f=/z-?45|7S-?7Q|1e|5y|7O-?1u/i;F b&&b.1b==3x&&d=="34"&&!f.1d(a)?b+"4t":b},18:{1r:9(b,c){C.J((c||"").2M(/\\s+/),9(i,a){6(!C.18.2N(b.18,a))b.18+=(b.18?" ":"")+a})},23:9(b,c){b.18=c!=Q?C.2B(b.18.2M(/\\s+/),9(a){F!C.18.2N(c,a)}).5w(" "):""},2N:9(t,c){F C.4K(c,(t.18||t).3s().2M(/\\s+/))>-1}},1V:9(e,o,f){I(E i 17 o){e.S["2U"+i]=e.S[i];e.S[i]=o[i]}f.T(e,[]);I(E i 17 o)e.S[i]=e.S["2U"+i]},1h:9(e,p){6(p=="1u"||p=="24"){E b={},3p,3o,d=["7J","7G","7F","7B"];C.J(d,9(){b["7A"+7]=0;b["7x"+7+"7u"]=0});C.1V(e,b,9(){6(C(e).3y(\':4N\')){3p=e.7t;3o=e.7q}G{e=C(e.3B(O)).1F(":4e").5d("27").3L().1h({3V:"1C",3k:"7n",11:"2m",7h:"0",7e:"0"}).57(e.P)[0];E a=C.1h(e.P,"3k")||"3g";6(a=="3g")e.P.S.3k="76";3p=e.74;3o=e.71;6(a=="3g")e.P.S.3k="3g";e.P.3e(e)}});F p=="1u"?3p:3o}F C.34(e,p)},34:9(h,d,g){E i,1R=[],1V=[];9 2E(a){6(!C.N.1H)F M;E b=R.2L.3b(a,K);F!b||b.44("2E")==""}6(d=="1e"&&C.N.12){i=C.1j(h.S,"1e");F i==""?"1":i}6(d.2k(/3a/i))d=x;6(!g&&h.S[d])i=h.S[d];G 6(R.2L&&R.2L.3b){6(d.2k(/3a/i))d="3a";d=d.1v(/([A-Z])/g,"-$1").2D();E e=R.2L.3b(h,K);6(e&&!2E(h))i=e.44(d);G{I(E a=h;a&&2E(a);a=a.P)1R.42(a);I(a=0;a<1R.H;a++)6(2E(1R[a])){1V[a]=1R[a].S.11;1R[a].S.11="2m"}i=d=="11"&&1V[1R.H-1]!=K?"1T":R.2L.3b(h,K).44(d)||"";I(a=0;a<1V.H;a++)6(1V[a]!=K)1R[a].S.11=1V[a]}6(d=="1e"&&i=="")i="1"}G 6(h.41){E f=d.1v(/\\-(\\w)/g,9(m,c){F c.1I()});i=h.41[d]||h.41[f]}F i},3c:9(a,c){E r=[];c=c||R;C.J(a,9(i,b){6(!b)F;6(b.1b==3x)b=b.3s();6(1f b=="1E"){E s=C.2s(b).2D(),1m=c.6a("1m"),1P=[];E a=!s.U("<1Z")&&[1,"<2b>","</2b>"]||!s.U("<6L")&&[1,"<4V>","</4V>"]||s.2k(/^<(6I|1w|6H|6F|6D)/)&&[1,"<1A>","</1A>"]||!s.U("<3O")&&[2,"<1A><1w>","</1w></1A>"]||(!s.U("<6A")||!s.U("<6y"))&&[3,"<1A><1w><3O>","</3O></1w></1A>"]||!s.U("<6x")&&[2,"<1A><1w></1w><4T>","</4T></1A>"]||C.N.12&&[1,"1m<1m>","</1m>"]||[0,"",""];1m.2W=a[1]+b+a[2];20(a[0]--)1m=1m.3Y;6(C.N.12){6(!s.U("<1A")&&s.U("<1w")<0)1P=1m.1k&&1m.1k.2Z;G 6(a[1]=="<1A>"&&s.U("<1w")<0)1P=1m.2Z;I(E n=1P.H-1;n>=0;--n)6(C.W(1P[n],"1w")&&!1P[n].2Z.H)1P[n].P.3e(1P[n]);6(/^\\s/.1d(b))1m.2p(c.60(b.2k(/^\\s*/)[0]),1m.1k)}b=C.2V(1m.2Z)}6(0===b.H&&(!C.W(b,"38")&&!C.W(b,"2b")))F;6(b[0]==Q||C.W(b,"38")||b.6u)r.Y(b);G r=C.29(r,b)});F r},1j:9(c,d,a){E e=C.3E(c)?{}:C.4q;6(d=="28"&&C.N.1H)c.P.3j;6(e[d]){6(a!=Q)c[e[d]]=a;F c[e[d]]}G 6(C.N.12&&d=="S")F C.1j(c.S,"6p",a);G 6(a==Q&&C.N.12&&C.W(c,"38")&&(d=="6n"||d=="6m"))F c.6k(d).5S;G 6(c.4y){6(a!=Q)c.6j(d,a);6(C.N.12&&/5R|32/.1d(d)&&!C.3E(c))F c.3F(d,2);F c.3F(d)}G{6(d=="1e"&&C.N.12){6(a!=Q){c.5y=1;c.1l=(c.1l||"").1v(/5T\\([^)]*\\)/,"")+(3m(a).3s()=="6d"?"":"5T(1e="+a*6c+")")}F c.1l?(3m(c.1l.2k(/1e=([^)]*)/)[1])/6c).3s():""}d=d.1v(/-([a-z])/8I,9(z,b){F b.1I()});6(a!=Q)c[d]=a;F c[d]}},2s:9(t){F(t||"").1v(/^\\s+|\\s+$/g,"")},2V:9(a){E r=[];6(1f a!="8H")I(E i=0,1M=a.H;i<1M;i++)r.Y(a[i]);G r=a.3S(0);F r},4K:9(b,a){I(E i=0,1M=a.H;i<1M;i++)6(a[i]==b)F i;F-1},29:9(a,b){6(C.N.12){I(E i=0;b[i];i++)6(b[i].1t!=8)a.Y(b[i])}G I(E i=0;b[i];i++)a.Y(b[i]);F a},4d:9(a){E r=[],4O=C.1q++;2g{I(E i=0,69=a.H;i<69;i++)6(4O!=a[i].1q){a[i].1q=4O;r.Y(a[i])}}2h(e){r=a}F r},1q:0,2B:9(b,a,c){6(1f a=="1E")a=2T("M||9(a,i){F "+a+"}");E d=[];I(E i=0,3P=b.H;i<3P;i++)6(!c&&a(b[i],i)||c&&!a(b[i],i))d.Y(b[i]);F d},3M:9(c,b){6(1f b=="1E")b=2T("M||9(a){F "+b+"}");E d=[];I(E i=0,3P=c.H;i<3P;i++){E a=b(c[i],i);6(a!==K&&a!=Q){6(a.1b!=1K)a=[a];d=d.8x(a)}}F d}});E u=8w.8u.2D();C.N={6b:(u.2k(/.+(?:8s|8q|8p|8o)[\\/: ]([\\d.]+)/)||[])[1],1H:/61/.1d(u),2t:/2t/.1d(u),12:/12/.1d(u)&&!/2t/.1d(u),3J:/3J/.1d(u)&&!/(8n|61)/.1d(u)};E x=C.N.12?"3I":"4G";C.14({8m:!C.N.12||R.8l=="8k",3I:C.N.12?"3I":"4G",4q:{"I":"8j","8i":"18","3a":x,4G:x,3I:x,2W:"2W",18:"18",2A:"2A",30:"30",27:"27",8h:"8g",28:"28",8f:"8e"}});C.J({5Y:"a.P",4C:"16.4C(a)",8c:"16.25(a,2,\'2a\')",8b:"16.25(a,2,\'4B\')",88:"16.4A(a.P.1k,a)",87:"16.4A(a.1k)"},9(i,n){C.15[i]=9(a){E b=C.3M(7,n);6(a&&1f a=="1E")b=C.2R(a,b);F 7.1O(C.4d(b))}});C.J({57:"3H",86:"5v",2p:"5u",85:"5t"},9(i,n){C.15[i]=9(){E a=1a;F 7.J(9(){I(E j=0,1M=a.H;j<1M;j++)C(a[j])[n](7)})}});C.J({5d:9(a){C.1j(7,a,"");7.84(a)},83:9(c){C.18.1r(7,c)},81:9(c){C.18.23(7,c)},80:9(c){C.18[C.18.2N(7,c)?"23":"1r"](7,c)},23:9(a){6(!a||C.1l(a,[7]).r.H)7.P.3e(7)},3K:9(){20(7.1k)7.3e(7.1k)}},9(i,n){C.15[i]=9(){F 7.J(n,1a)}});C.J(["5Q","5P","5M","5L"],9(i,n){C.15[n]=9(a,b){F 7.1l(":"+n+"("+a+")",b)}});C.J(["1u","24"],9(i,n){C.15[n]=9(h){F h==Q?(7.H?C.1h(7[0],n):K):7.1h(n,h.1b==3n?h:h+"4t")}});E A=C.N.1H&&5K(C.N.6b)<7Z?"(?:[\\\\w*2l-]|\\\\\\\\.)":"(?:[\\\\w\\7Y-\\7V*2l-]|\\\\\\\\.)",5I=1s 3C("^[/>]\\\\s*("+A+"+)"),5H=1s 3C("^("+A+"+)(#)("+A+"+)"),5G=1s 3C("^([#.]?)("+A+"*)");C.14({4w:{"":"m[2]==\'*\'||16.W(a,m[2])","#":"a.3F(\'2j\')==m[2]",":":{5P:"i<m[3]-0",5M:"i>m[3]-0",25:"m[3]-0==i",5Q:"m[3]-0==i",2H:"i==0",2P:"i==r.H-1",5E:"i%2==0",5D:"i%2","2H-3z":"a.P.4L(\'*\')[0]==a","2P-3z":"16.25(a.P.3Y,1,\'4B\')==a","7U-3z":"!16.25(a.P.3Y,2,\'4B\')",5Y:"a.1k",3K:"!a.1k",5L:"(a.5Z||a.7T||\'\').U(m[3])>=0",4N:\'"1C"!=a.L&&16.1h(a,"11")!="1T"&&16.1h(a,"3V")!="1C"\',1C:\'"1C"==a.L||16.1h(a,"11")=="1T"||16.1h(a,"3V")=="1C"\',7R:"!a.30",30:"a.30",27:"a.27",28:"a.28||16.1j(a,\'28\')",2Q:"\'2Q\'==a.L",4e:"\'4e\'==a.L",3i:"\'3i\'==a.L",4v:"\'4v\'==a.L",5C:"\'5C\'==a.L",4u:"\'4u\'==a.L",5B:"\'5B\'==a.L",5A:"\'5A\'==a.L",1X:\'"1X"==a.L||16.W(a,"1X")\',39:"/39|2b|7P|1X/i.1d(a.W)",2N:"16.1F(m[3],a).H"},"[":"16.1F(m[2],a).H"},5x:[/^\\[ *(@)([\\w-]+) *([!*$^~=]*) *(\'?"?)(.*?)\\4 *\\]/,/^(\\[)\\s*(.*?(\\[.*?\\])?[^[]*?)\\s*\\]/,/^(:)([\\w-]+)\\("?\'?(.*?(\\(.*?\\))?[^(]*?)"?\'?\\)/,1s 3C("^([:.#]*)("+A+"+)")],2R:9(a,c,b){E d,1Y=[];20(a&&a!=d){d=a;E f=C.1l(a,c,b);a=f.t.1v(/^\\s*,\\s*/,"");1Y=b?c=f.r:C.29(1Y,f.r)}F 1Y},1F:9(t,l){6(1f t!="1E")F[t];6(l&&!l.1t)l=K;l=l||R;6(!t.U("//")){t=t.2K(2,t.H)}G 6(!t.U("/")&&!l.2I){l=l.3D;t=t.2K(1,t.H);6(t.U("/")>=1)t=t.2K(t.U("/"),t.H)}E d=[l],2q=[],2P;20(t&&2P!=t){E r=[];2P=t;t=C.2s(t).1v(/^\\/\\//,"");E k=M;E g=5I;E m=g.2d(t);6(m){E o=m[1].1I();I(E i=0;d[i];i++)I(E c=d[i].1k;c;c=c.2a)6(c.1t==1&&(o=="*"||c.W.1I()==o.1I()))r.Y(c);d=r;t=t.1v(g,"");6(t.U(" ")==0)5X;k=O}G{g=/^((\\/?\\.\\.)|([>\\/+~]))\\s*(\\w*)/i;6((m=g.2d(t))!=K){r=[];E o=m[4],1q=C.1q++;m=m[1];I(E j=0,2o=d.H;j<2o;j++)6(m.U("..")<0){E n=m=="~"||m=="+"?d[j].2a:d[j].1k;I(;n;n=n.2a)6(n.1t==1){6(m=="~"&&n.1q==1q)1J;6(!o||n.W.1I()==o.1I()){6(m=="~")n.1q=1q;r.Y(n)}6(m=="+")1J}}G r.Y(d[j].P);d=r;t=C.2s(t.1v(g,""));k=O}}6(t&&!k){6(!t.U(",")){6(l==d[0])d.4s();2q=C.29(2q,d);r=d=[l];t=" "+t.2K(1,t.H)}G{E h=5H;E m=h.2d(t);6(m){m=[0,m[2],m[3],m[1]]}G{h=5G;m=h.2d(t)}m[2]=m[2].1v(/\\\\/g,"");E f=d[d.H-1];6(m[1]=="#"&&f&&f.37&&!C.3E(f)){E p=f.37(m[2]);6((C.N.12||C.N.2t)&&p&&1f p.2j=="1E"&&p.2j!=m[2])p=C(\'[@2j="\'+m[2]+\'"]\',f)[0];d=r=p&&(!m[3]||C.W(p,m[3]))?[p]:[]}G{I(E i=0;d[i];i++){E a=m[1]!=""||m[0]==""?"*":m[2];6(a=="*"&&d[i].W.2D()=="4P")a="2O";r=C.29(r,d[i].4L(a))}6(m[1]==".")r=C.4r(r,m[2]);6(m[1]=="#"){E e=[];I(E i=0;r[i];i++)6(r[i].3F("2j")==m[2]){e=[r[i]];1J}r=e}d=r}t=t.1v(h,"")}}6(t){E b=C.1l(t,r);d=r=b.r;t=C.2s(b.t)}}6(t)d=[];6(d&&l==d[0])d.4s();2q=C.29(2q,d);F 2q},4r:9(r,m,a){m=" "+m+" ";E c=[];I(E i=0;r[i];i++){E b=(" "+r[i].18+" ").U(m)>=0;6(!a&&b||a&&!b)c.Y(r[i])}F c},1l:9(t,r,h){E d;20(t&&t!=d){d=t;E p=C.5x,m;I(E i=0;p[i];i++){m=p[i].2d(t);6(m){t=t.7N(m[0].H);m[2]=m[2].1v(/\\\\/g,"");1J}}6(!m)1J;6(m[1]==":"&&m[2]=="5l")r=C.1l(m[3],r,O).r;G 6(m[1]==".")r=C.4r(r,m[2],h);G 6(m[1]=="@"){E g=[],L=m[3];I(E i=0,2o=r.H;i<2o;i++){E a=r[i],z=a[C.4q[m[2]]||m[2]];6(z==K||/5R|32|28/.1d(m[2]))z=C.1j(a,m[2])||\'\';6((L==""&&!!z||L=="="&&z==m[5]||L=="!="&&z!=m[5]||L=="^="&&z&&!z.U(m[5])||L=="$="&&z.2K(z.H-m[5].H)==m[5]||(L=="*="||L=="~=")&&z.U(m[5])>=0)^h)g.Y(a)}r=g}G 6(m[1]==":"&&m[2]=="25-3z"){E e=C.1q++,g=[],1d=/(\\d*)n\\+?(\\d*)/.2d(m[3]=="5E"&&"2n"||m[3]=="5D"&&"2n+1"||!/\\D/.1d(m[3])&&"n+"+m[3]||m[3]),2H=(1d[1]||1)-0,d=1d[2]-0;I(E i=0,2o=r.H;i<2o;i++){E j=r[i],P=j.P;6(e!=P.1q){E c=1;I(E n=P.1k;n;n=n.2a)6(n.1t==1)n.4o=c++;P.1q=e}E b=M;6(2H==1){6(d==0||j.4o==d)b=O}G 6((j.4o+d)%2H==0)b=O;6(b^h)g.Y(j)}r=g}G{E f=C.4w[m[1]];6(1f f!="1E")f=C.4w[m[1]][m[2]];f=2T("M||9(a,i){F "+f+"}");r=C.2B(r,f,h)}}F{r:r,t:t}},4C:9(c){E b=[];E a=c.P;20(a&&a!=R){b.Y(a);a=a.P}F b},25:9(a,e,c,b){e=e||1;E d=0;I(;a;a=a[c])6(a.1t==1&&++d==e)1J;F a},4A:9(n,a){E r=[];I(;n;n=n.2a){6(n.1t==1&&(!a||n!=a))r.Y(n)}F r}});C.1c={1r:9(f,d,c,b){6(C.N.12&&f.3t!=Q)f=19;6(!c.22)c.22=7.22++;6(b!=Q){E e=c;c=9(){F e.T(7,1a)};c.V=b;c.22=e.22}6(!f.$1i)f.$1i={};6(!f.$1y)f.$1y=9(){E a;6(1f C=="Q"||C.1c.4n)F a;a=C.1c.1y.T(f,1a);F a};E g=f.$1i[d];6(!g){g=f.$1i[d]={};6(f.4m)f.4m(d,f.$1y,M);G f.7M("3r"+d,f.$1y)}g[c.22]=c;7.1D[d]=O},22:1,1D:{},23:9(c,b,a){E d=c.$1i,2c,45;6(d){6(b&&b.L){a=b.4l;b=b.L}6(!b){I(b 17 d)7.23(c,b)}G 6(d[b]){6(a)4k d[b][a.22];G I(a 17 c.$1i[b])4k d[b][a];I(2c 17 d[b])1J;6(!2c){6(c.4j)c.4j(b,c.$1y,M);G c.7L("3r"+b,c.$1y);2c=K;4k d[b]}}I(2c 17 d)1J;6(!2c)c.$1y=c.$1i=K}},1z:9(c,b,d){b=C.2V(b||[]);6(!d){6(7.1D[c])C("*").1r([19,R]).1z(c,b)}G{E a,2c,15=C.1g(d[c]||K);b.42(7.4i({L:c,1S:d}));6(C.1g(d.$1y))a=d.$1y.T(d,b);6(!15&&d["3r"+c]&&d["3r"+c].T(d,b)===M)a=M;6(15&&a!==M&&!(C.W(d,\'a\')&&c=="4h")){7.4n=O;d[c]()}7.4n=M}},1y:9(b){E a;b=C.1c.4i(b||19.1c||{});E c=7.$1i&&7.$1i[b.L],2e=1K.3v.3S.2S(1a,1);2e.42(b);I(E j 17 c){2e[0].4l=c[j];2e[0].V=c[j].V;6(c[j].T(7,2e)===M){b.2u();b.2X();a=M}}6(C.N.12)b.1S=b.2u=b.2X=b.4l=b.V=K;F a},4i:9(c){E a=c;c=C.14({},a);c.2u=9(){6(a.2u)a.2u();a.7I=M};c.2X=9(){6(a.2X)a.2X();a.7H=O};6(!c.1S&&c.5r)c.1S=c.5r;6(C.N.1H&&c.1S.1t==3)c.1S=a.1S.P;6(!c.4g&&c.4F)c.4g=c.4F==c.1S?c.7C:c.4F;6(c.5p==K&&c.66!=K){E e=R.3D,b=R.4z;c.5p=c.66+(e&&e.5o||b.5o||0);c.7z=c.7v+(e&&e.5m||b.5m||0)}6(!c.3Q&&(c.5k||c.5j))c.3Q=c.5k||c.5j;6(!c.5i&&c.5g)c.5i=c.5g;6(!c.3Q&&c.1X)c.3Q=(c.1X&1?1:(c.1X&2?3:(c.1X&4?2:0)));F c}};C.15.14({3l:9(c,a,b){F c=="5f"?7.5e(c,a,b):7.J(9(){C.1c.1r(7,c,b||a,b&&a)})},5e:9(d,b,c){F 7.J(9(){C.1c.1r(7,d,9(a){C(7).49(a);F(c||b).T(7,1a)},c&&b)})},49:9(a,b){F 7.J(9(){C.1c.23(7,a,b)})},1z:9(a,b){F 7.J(9(){C.1c.1z(a,b,7)})},1W:9(){E a=1a;F 7.4h(9(e){7.3T=0==7.3T?1:0;e.2u();F a[7.3T].T(7,[e])||M})},7p:9(f,g){9 3U(e){E p=e.4g;20(p&&p!=7)2g{p=p.P}2h(e){p=7};6(p==7)F M;F(e.L=="3W"?f:g).T(7,[e])}F 7.3W(3U).5b(3U)},1L:9(f){5a();6(C.36)f.T(R,[C]);G C.2C.Y(9(){F f.T(7,[C])});F 7}});C.14({36:M,2C:[],1L:9(){6(!C.36){C.36=O;6(C.2C){C.J(C.2C,9(){7.T(R)});C.2C=K}6(C.N.3J||C.N.2t)R.4j("59",C.1L,M);6(!19.7m.H)C(19).2f(9(){C("#4b").23()})}}});C.J(("7l,7k,2f,7j,7i,5f,4h,7g,"+"7f,7d,7c,3W,5b,7b,2b,"+"4u,7a,79,78,3f").2M(","),9(i,o){C.15[o]=9(f){F f?7.3l(o,f):7.1z(o)}});E w=M;9 5a(){6(w)F;w=O;6(C.N.3J||C.N.2t)R.4m("59",C.1L,M);G 6(C.N.12){R.75("<73"+"72 2j=4b 70=O "+"32=//:><\\/33>");E a=R.37("4b");6(a)a.6Z=9(){6(R.3d!="1x")F;C.1L()};a=K}G 6(C.N.1H)C.48=3t(9(){6(R.3d=="6Y"||R.3d=="1x"){47(C.48);C.48=K;C.1L()}},10);C.1c.1r(19,"2f",C.1L)}C.15.14({6X:9(c,b,a){7.2f(c,b,a,1)},2f:9(g,e,c,d){6(C.1g(g))F 7.3l("2f",g);c=c||9(){};E f="46";6(e)6(C.1g(e)){c=e;e=K}G{e=C.2O(e);f="55"}E h=7;C.31({1G:g,L:f,V:e,2F:d,1x:9(a,b){6(b=="1U"||!d&&b=="54")h.5W(a.43);4x(9(){h.J(c,[a.43,b,a])},13)}});F 7},6W:9(){F C.2O(7)},6V:9(){}});C.J("53,52,51,50,4Z,5h".2M(","),9(i,o){C.15[o]=9(f){F 7.3l(o,f)}});C.14({21:9(e,c,a,d,b){6(C.1g(c)){a=c;c=K}F C.31({L:"46",1G:e,V:c,1U:a,3G:d,2F:b})},6U:9(d,b,a,c){F C.21(d,b,a,c,1)},6T:9(b,a){F C.21(b,K,a,"33")},77:9(c,b,a){F C.21(c,b,a,"56")},6S:9(d,b,a,c){6(C.1g(b)){a=b;b={}}F C.31({L:"55",1G:d,V:b,1U:a,3G:c})},6R:9(a){C.3u.1Q=a},6Q:9(a){C.14(C.3u,a)},3u:{1D:O,L:"46",1Q:0,4Y:"6P/x-6O-38-6N",4X:O,2w:O,V:K},3h:{},31:9(s){s=C.14(O,s,C.14(O,{},C.3u,s));6(s.V){6(s.4X&&1f s.V!="1E")s.V=C.2O(s.V);6(s.L.2D()=="21"){s.1G+=(s.1G.U("?")>-1?"&":"?")+s.V;s.V=K}}6(s.1D&&!C.40++)C.1c.1z("53");E f=M;E h=19.4W?1s 4W("6M.6K"):1s 58();h.6J(s.L,s.1G,s.2w);6(s.V)h.4c("7r-7s",s.4Y);6(s.2F)h.4c("6G-3Z-6E",C.3h[s.1G]||"7w, 6C 7y 6B 4J:4J:4J 6z");h.4c("X-7D-7E","58");6(s.4U)s.4U(h);6(s.1D)C.1c.1z("5h",[h,s]);E g=9(d){6(!f&&h&&(h.3d==4||d=="1Q")){f=O;6(i){47(i);i=K}E c=d=="1Q"&&"1Q"||!C.5n(h)&&"3f"||s.2F&&C.5s(h,s.1G)&&"54"||"1U";6(c=="1U"){2g{E a=C.5q(h,s.3G)}2h(e){c="4I"}}6(c=="1U"){E b;2g{b=h.4f("4S-3Z")}2h(e){}6(s.2F&&b)C.3h[s.1G]=b;6(s.1U)s.1U(a,c);6(s.1D)C.1c.1z("4Z",[h,s])}G C.3X(s,h,c);6(s.1D)C.1c.1z("51",[h,s]);6(s.1D&&!--C.40)C.1c.1z("52");6(s.1x)s.1x(h,c);6(s.2w)h=K}};6(s.2w){E i=3t(g,13);6(s.1Q>0)4x(9(){6(h){h.6w();6(!f)g("1Q")}},s.1Q)}2g{h.6v(s.V)}2h(e){C.3X(s,h,K,e)}6(!s.2w)g();F h},3X:9(s,a,b,e){6(s.3f)s.3f(a,b,e);6(s.1D)C.1c.1z("50",[a,s,e])},40:0,5n:9(r){2g{F!r.26&&6t.6r=="4v:"||(r.26>=4R&&r.26<6q)||r.26==5z||C.N.1H&&r.26==Q}2h(e){}F M},5s:9(a,c){2g{E b=a.4f("4S-3Z");F a.26==5z||b==C.3h[c]||C.N.1H&&a.26==Q}2h(e){}F M},5q:9(r,a){E b=r.4f("6o-L");E c=a=="5F"||!a&&b&&b.U("5F")>=0;V=c?r.7W:r.43;6(c&&V.3D.4y=="4I")7X"4I";6(a=="33")C.4E(V);6(a=="56")V=2T("("+V+")");F V},2O:9(a){E s=[];6(a.1b==1K||a.3w)C.J(a,9(){s.Y(2y(7.6l)+"="+2y(7.2A))});G I(E j 17 a)6(a[j]&&a[j].1b==1K)C.J(a[j],9(){s.Y(2y(j)+"="+2y(7))});G s.Y(2y(j)+"="+2y(a[j]));F s.5w("&")}});C.15.14({1o:9(b,a){F b?7.1B({1u:"1o",24:"1o",1e:"1o"},b,a):7.1l(":1C").J(9(){7.S.11=7.2r?7.2r:"";6(C.1h(7,"11")=="1T")7.S.11="2m"}).3L()},1p:9(b,a){F b?7.1B({1u:"1p",24:"1p",1e:"1p"},b,a):7.1l(":4N").J(9(){7.2r=7.2r||C.1h(7,"11");6(7.2r=="1T")7.2r="2m";7.S.11="1T"}).3L()},5O:C.15.1W,1W:9(a,b){F C.1g(a)&&C.1g(b)?7.5O(a,b):a?7.1B({1u:"1W",24:"1W",1e:"1W"},a,b):7.J(9(){C(7)[C(7).3y(":1C")?"1o":"1p"]()})},6i:9(b,a){F 7.1B({1u:"1o"},b,a)},6h:9(b,a){F 7.1B({1u:"1p"},b,a)},6g:9(b,a){F 7.1B({1u:"1W"},b,a)},6f:9(b,a){F 7.1B({1e:"1o"},b,a)},89:9(b,a){F 7.1B({1e:"1p"},b,a)},6e:9(c,a,b){F 7.1B({1e:a},c,b)},1B:9(d,h,f,g){F 7.1n(9(){E c=C(7).3y(":1C"),1Z=C.5V(h,f,g),5U=7;I(E p 17 d){6(d[p]=="1p"&&c||d[p]=="1o"&&!c)F C.1g(1Z.1x)&&1Z.1x.T(7);6(p=="1u"||p=="24"){1Z.11=C.1h(7,"11");1Z.2z=7.S.2z}}6(1Z.2z!=K)7.S.2z="1C";7.2v=C.14({},d);C.J(d,9(a,b){E e=1s C.2Y(5U,1Z,a);6(b.1b==3x)e.3R(e.1Y()||0,b);G e[b=="1W"?c?"1o":"1p":b](d)});F O})},1n:9(a,b){6(!b){b=a;a="2Y"}F 7.J(9(){6(!7.1n)7.1n={};6(!7.1n[a])7.1n[a]=[];7.1n[a].Y(b);6(7.1n[a].H==1)b.T(7)})}});C.14({5V:9(b,a,c){E d=b&&b.1b==8G?b:{1x:c||!c&&a||C.1g(b)&&b,1N:b,35:c&&a||a&&a.1b!=8F&&a};d.1N=(d.1N&&d.1N.1b==3x?d.1N:{8D:8C,8B:4R}[d.1N])||8A;d.2U=d.1x;d.1x=9(){C.68(7,"2Y");6(C.1g(d.2U))d.2U.T(7)};F d},35:{62:9(p,n,b,a){F b+a*p},4H:9(p,n,b,a){F((-67.8z(p*67.8y)/2)+0.5)*a+b}},1n:{},68:9(b,a){a=a||"2Y";6(b.1n&&b.1n[a]){b.1n[a].4s();E f=b.1n[a][0];6(f)f.T(b)}},3N:[],2Y:9(f,e,g){E z=7;E y=f.S;z.a=9(){6(e.3q)e.3q.T(f,[z.2x]);6(g=="1e")C.1j(y,"1e",z.2x);G{y[g]=5K(z.2x)+"4t";6(g=="1u"||g=="24")y.11="2m"}};z.65=9(){F 3m(C.1h(f,g))};z.1Y=9(){E r=3m(C.34(f,g));F r&&r>-8v?r:z.65()};z.3R=9(c,b){z.4M=(1s 64()).63();z.2x=c;z.a();C.3N.Y(9(){F z.3q(c,b)});6(C.3N.H==1){E d=3t(9(){E a=C.3N;I(E i=0;i<a.H;i++)6(!a[i]())a.8t(i--,1);6(!a.H)47(d)},13)}};z.1o=9(){6(!f.2i)f.2i={};f.2i[g]=C.1j(f.S,g);e.1o=O;z.3R(0,7.1Y());6(g!="1e")y[g]="8r";C(f).1o()};z.1p=9(){6(!f.2i)f.2i={};f.2i[g]=C.1j(f.S,g);e.1p=O;z.3R(7.1Y(),0)};z.3q=9(a,c){E t=(1s 64()).63();6(t>e.1N+z.4M){z.2x=c;z.a();6(f.2v)f.2v[g]=O;E b=O;I(E i 17 f.2v)6(f.2v[i]!==O)b=M;6(b){6(e.11!=K){y.2z=e.2z;y.11=e.11;6(C.1h(f,"11")=="1T")y.11="2m"}6(e.1p)y.11="1T";6(e.1p||e.1o)I(E p 17 f.2v)C.1j(y,p,f.2i[p])}6(b&&C.1g(e.1x))e.1x.T(f);F M}G{E n=t-7.4M;E p=n/e.1N;z.2x=C.35[e.35||(C.35.4H?"4H":"62")](p,n,a,(c-a),e.1N);z.a()}F O}}})})();',62,541,'||||||if|this||function|||||||||||||||||||||||||||||||var|return|else|length|for|each|null|type|false|browser|true|parentNode|undefined|document|style|apply|indexOf|data|nodeName||push|||display|msie||extend|fn|jQuery|in|className|window|arguments|constructor|event|test|opacity|typeof|isFunction|css|events|attr|firstChild|filter|div|queue|show|hide|mergeNum|add|new|nodeType|height|replace|tbody|complete|handle|trigger|table|animate|hidden|global|string|find|url|safari|toUpperCase|break|Array|ready|al|duration|pushStack|tb|timeout|stack|target|none|success|swap|toggle|button|cur|opt|while|get|guid|remove|width|nth|status|checked|selected|merge|nextSibling|select|ret|exec|args|load|try|catch|orig|id|match|_|block||rl|insertBefore|done|oldblock|trim|opera|preventDefault|curAnim|async|now|encodeURIComponent|overflow|value|grep|readyList|toLowerCase|color|ifModified|val|first|ownerDocument|domManip|substr|defaultView|split|has|param|last|text|multiFilter|call|eval|old|makeArray|innerHTML|stopPropagation|fx|childNodes|disabled|ajax|src|script|curCSS|easing|isReady|getElementById|form|input|float|getComputedStyle|clean|readyState|removeChild|error|static|lastModified|checkbox|selectedIndex|position|bind|parseFloat|String|oWidth|oHeight|step|on|toString|setInterval|ajaxSettings|prototype|jquery|Number|is|child|ol|cloneNode|RegExp|documentElement|isXMLDoc|getAttribute|dataType|append|styleFloat|mozilla|empty|end|map|timers|tr|el|which|custom|slice|lastToggle|handleHover|visibility|mouseover|handleError|lastChild|Modified|active|currentStyle|unshift|responseText|getPropertyValue|index|GET|clearInterval|safariTimer|unbind|init|__ie_init|setRequestHeader|unique|radio|getResponseHeader|relatedTarget|click|fix|removeEventListener|delete|handler|addEventListener|triggered|nodeIndex|appendChild|props|classFilter|shift|px|submit|file|expr|setTimeout|tagName|body|sibling|previousSibling|parents|deep|globalEval|fromElement|cssFloat|swing|parsererror|00|inArray|getElementsByTagName|startTime|visible|num|object|prop|200|Last|colgroup|beforeSend|fieldset|ActiveXObject|processData|contentType|ajaxSuccess|ajaxError|ajaxComplete|ajaxStop|ajaxStart|notmodified|POST|json|appendTo|XMLHttpRequest|DOMContentLoaded|bindReady|mouseout|prevObject|removeAttr|one|unload|ctrlKey|ajaxSend|metaKey|keyCode|charCode|not|scrollTop|httpSuccess|scrollLeft|pageX|httpData|srcElement|httpNotModified|after|before|prepend|join|parse|zoom|304|reset|image|password|odd|even|xml|quickClass|quickID|quickChild|setArray|parseInt|contains|gt|execScript|_toggle|lt|eq|href|nodeValue|alpha|self|speed|html|continue|parent|textContent|createTextNode|webkit|linear|getTime|Date|max|clientX|Math|dequeue|fl|createElement|version|100|NaN|fadeTo|fadeIn|slideToggle|slideUp|slideDown|setAttribute|getAttributeNode|name|method|action|content|cssText|300|protocol|FORM|location|options|send|abort|col|th|GMT|td|1970|01|cap|Since|colg|If|tfoot|thead|open|XMLHTTP|leg|Microsoft|urlencoded|www|application|ajaxSetup|ajaxTimeout|post|getScript|getIfModified|evalScripts|serialize|loadIfModified|loaded|onreadystatechange|defer|clientWidth|ipt|scr|clientHeight|write|relative|getJSON|keyup|keypress|keydown|change|mousemove|mouseup|left|mousedown|dblclick|right|scroll|resize|focus|blur|frames|absolute|clone|hover|offsetWidth|Content|Type|offsetHeight|Width|clientY|Thu|border|Jan|pageY|padding|Left|toElement|Requested|With|Right|Bottom|cancelBubble|returnValue|Top|size|detachEvent|attachEvent|substring|line|textarea|weight|enabled|font|innerText|only|uFFFF|responseXML|throw|u0128|417|toggleClass|removeClass|wrap|addClass|removeAttribute|insertAfter|prependTo|children|siblings|fadeOut|noConflict|prev|next|Boolean|maxLength|maxlength|readOnly|readonly|class|htmlFor|CSS1Compat|compatMode|boxModel|compatible|ie|ra|it|1px|rv|splice|userAgent|10000|navigator|concat|PI|cos|400|fast|600|slow|reverse|Function|Object|array|ig'.split('|'),0,{}))
\ No newline at end of file
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/static/layout.css
@@ -0,0 +1,42 @@
+html, body { margin: 0; padding: 0; }
+body { background: #ddd; color: #333; font: normal 90%/1.3 Arial,Helvetica,sans-serif; }
+:link, :visited { color: #c10000; text-decoration: none; }
+:link:hover, :visited:hover { text-decoration: underline; }
+:link img, :visited img { border: none; }
+h1 { color: #666;
+  font: normal xx-large/1.5 Georgia,serif; margin: 0 0 .5em;
+}
+blockquote { font-style: italic; }
+form table { margin-bottom: 1em; }
+form table tbody th { font-weight: normal; padding-top: .3em; text-align: right;
+  vertical-align: top;
+}
+form label { color: #666; font-size: 90%; }
+
+#wrap { background: #fff; width: 600px; margin: 30px auto; }
+#content { border-left: 10px solid #b00; min-height: 240px; padding: 10px; }
+#header img { float: right; margin-top: -21px; margin-right: -37px; }
+
+#footer { background: #4A4D4D; border-top: 2px solid #333; font-size: x-small;
+  padding: 3px; text-align: center;
+}
+#footer hr { display: none; }
+.legalese { color: #999; margin: 0; }
+
+ol.links li .info { font-size: 85%; }
+ol.links li .info :link, ol.links li .info :visited { color: #666; }
+
+ul.comments { list-style: none; margin: 1em 0; padding: 0 0 0 1em; }
+ul.comments li { color: #999; margin: 0 0 1em; }
+ul.comments blockquote { color: #333; font-style: normal; margin: 0;
+  padding: 0;
+}
+.action:link, .action:visited { background: #f3f3f3; border: 1px outset #ddd;
+  color: #666; font-size: 90%; padding: 0 .3em;
+}
+.action:link:hover, .action:visited:hover { background: #e8e8e8;
+  border-color: #aaa; color: #000; text-decoration: none;
+}
+
+form p.hint { color: #666; font-size: 90%; font-style: italic; margin: 0; }
+form .error { color: #b00; }
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..60267e548999abf7ddb818be82c0e9397ef51b7b
GIT binary patch
literal 3266
zc$@*k3_bHlNk%w1Vd(%{0K@<Q0002*@9)ga%xi0FGcz--t*zbN-ISD+5D*YjQd0l_
z|Ad5uy}i9cLPFNo)*&GwW@cukrKNj&dk6>!#l^+7wY5r0N;*0^nVFg9=H^;jTKD(&
zjEszPb8{IP87V0#A^8LV2LS&7EC2ui0O<f*000I5pdpTAX`X1Ru59bRa4f@4Z3}pA
z*K_axz@TtQ9Nlurq%z4XI-k&}G*(+$uh!;t%k6qu4lI&VDVQq^ig@Yxy5DeOSWJmX
zI{3Vv!&o!hae;z^C39F55)c}T8W00|c@Q0aema1Jn3+?CSP=;p772|55(^v$sHhwZ
z5(5~OIBJ)gw6#B+R|gHdq8$qd5fKFy#Ki>>2fwfaX#%shB?1+~3_c7649`#wWfXCh
z6&Du`2@nz(!_7++*BBAY41L-`T0jgC4*>f56CE!O7!r8c2jGE*2u~1Qn6Pic1qs!<
zz-llq!2k{sn%OX6gI&gD!zj#yAz*<90tg8IBKUA{h5!u&V*>EDB4A3H`#cN~5Cjnc
zoCFCR5W?VR0-hvbMG$9^12$(ZYEUS!OhAMRB70!?36$W4kti7C>k6Pk3I-Z(l5Ge9
zsf+^`*2a7=)JFuh1d`%dK#Bu8N=Oo1umE9!urkONu#!;sgaZm{U{N@L6)cJfW98~g
z@Pfj?m<S$-eqbTRfwdkO%JhIRMgy`FKFe(27Dfcs`aT$`F#zZU1RjD<7(RS0*vC~P
zV5k5!Y66@N6D0T`6Tu7W`yMQ)y}|-ep-3ZmmnngUZw&=Rl+2vdEJ?MSMRe3mU*Cll
zxco@?*g~w#4+qR5K};UF(Az#=#Q*{SP79n+0apb1^MY(SMO4B9xaos{doC;(Q-cQN
zKtX+)AwW|K`K_@^0a>X~0ek>VP~Hh05Y(6o6yWxNWoN-e5QirCNDv4AU`CK`VX+5*
z2i&RfTucaYq9lni5n!PTDPmE<0F*W7AdCp4mqKp{+~_1hV!9L)kS2K8(3LC<xDS#j
z2%uyGFJK^(S)6d-!2=%<An2e3+}S}Xr%=Nq30Y#HfnhD0U?*s7o{)_T0^o+{4-{5f
z!jlNvcH|14#bl-maS`OrGN;+NC<|4+)By(B%<~#S6Qnsrjr+9b1B7Mar=+heFfi&0
z5zO_f0R_b5%reRTxU34X4gdlF2h5$YfJ~<bky(L2fEw%y&3g6Eu`7rR0FhC}R0S**
z#1yFyu}YwU1S~!y0E`+uVeCHr0^#Pk-Ngi_3Tn|wZH9Td5R(Cou0xOq^Y#$y0ts{w
zLB#_XH0=v=>VrWLArtIkOcJ($9RLce(D8{Y;0ke71U*o(OI3A{F|D&?8q>&6+^SEL
zZ~f>(s`@nC0@5v?6%)HGs5J93LES7vtPoh>%B3Ul8Y&bz3K6Y7o@y(z3*)i;0<ZzI
zfE&0~X=*hMu{NMJ&jI8efRYvH?f2h+2M+j%`y?Rv;e!9G4+Dt%?SQ+D>+L21k?ZX>
z01Z?gfLagaoxw7=X^_GH-x<gdu@!@D?SKNT3lLKStj8|<?6lWD``7?TVEgX28$gf%
zyw6S{1;NK2G6BWMt~krdD<Inkv2PHf7zS*hmhQ6$aCKD$kTYvG#>Z4Z^W=x$Yx%ne
z@H_cJm(@H6`yk)E?>RM}nf%e0{w`Fbvo0G4w9{S;uttvkh$MDMTha#5M?N6|fB_7w
z-3Hdzyn&4`d94Bf{*tGX=B;dl<dY2s0!W3bP+%O7$`|W4kRb>}uz3kw-3w<&!Q@>o
z0?pG?(#ofn33w1e5tI!Mvp2x13}7(V$`<Pgw+{%;FnAOSz!PPM8{jo>TjxvO*gzP(
z%=j>Q5P}^8GgATo68@tq;kdxgYQ;Jo@?(Kh++Bv^7`rNx4~zP^VDYwCzRVz?d0~uT
zW}GNH0Y;^6NL)a^48Xd}S%8kVo8UeW;D7`?AUK%JB;}H4$19?#d3+?F9|5_KM~-nH
z#3Lf6sv;Qg(F!T7b0GF0IlBztDnmFyfE0_@$>f!+j9QFe`DEh(QtHE$=*x^13+aN>
zDM15Fxz-{XsetzBk(RNeUwsG}yc!noa@^!yD4WMQV*(%nLEMK7nF$6~ii0qQ;Tko8
zwN7lZ?s4?l5c#|ryn!t<cy8n(?{Fi~+Cd1N43rHu*@>lPRDdE#Jcm3+z`B(IV3+o6
z9j-(H0Y{Yo0vHU)K>7H|J8>-(pu%jPB=sT2;vF=CY7wa_QM8N&h{JB?SdN#d$)1eL
z&UN*p<4OVW&)~HWj>XGhL2IW?0BFFR2t^-mWXe#-z{>(REyn_0`3%-U=#M_t=p-Sj
zzI_yxcPj%c22+|mCIX-X=6osx34+ns8B$V}puiEwVF3*cAf7uF0qcbJ#jxU&A7mw|
z9Opzn`0WssLy9W{%c_sMvhJ>W(T_MPVvYkmWgf94$O{D<yu<1PsZX?}I_F1_s1^^8
zOQqc}C5yhMT2=v=RgWKRnpF+>;wrL00qgL1Q3j^!0Krr3K2AErrq+{p<<iG2ix;vB
zkS}fj0ATL!ASBoF0rj+I%q$joTM@r*^|LBqUA&fCTYv?RvWugmX`j2h**4E>)@>d!
z$%nM=l5csay=*kZKwfhAl`$C+36EH(+y|mhaV=f!K1LYF?*f3nwgc8}cXuEF<hOXH
z^>6U{%fRRYAi%g4?-Jm0)4q}d7+49ca4WhG0G4lTAKo2%20%v(W0-ag&T;YFB;?&W
zR%HC$hYQh{K>*awkd5kIiw_Lr;JUzk=R_YKi%dK0#`eDi(QfUUjNAt9Bf`9c&;&MY
zJE1MPd?kCbe6FxpyEOvAF|G#-Vt3dHpqO_={$@<rT)y?{L(H^OGgXrZDm2^9!mcg<
zo|_psFga)Gp;<KtM)LeI(y}gX6gDT9XJ=vngnEW1Mzrh<4S+W{FKv#low!ojyh)?4
zeYe$~6<BO%U!X%4ou+8)nD~G{*KT=V#_j9kh&xrU(4s%yozP-EyX8@PcWHC=56v3D
z6!DfEo&9XENiaK5Y1?)gU{A6YkU*kN(T&*+-sO0VKBQgyvg|o)@Unf%2?uZq9LVZl
zfve{k)KV=FFgEKLKp_n#58jxY;*l@RJ9v9w(cAVg?>5lBa~IFt#`OyAriX+yde12n
z6$0%Tpd$?2mI3O@P4cW~swb8#oU$v~1ZX|X?Cj0}3A}*9cPAu|%Nt6w%MVunPtGD4
z`=E9Oz$IxG(Eu3L?0qqr05BuiJ{yx7SQLw3g8_hX1!Q0W@UZ})0j*>Px-~|pRssep
za*=5g!FDgAl0J%=aekB;eHqA5nIKaD4kQ3Z4Jf-Oy@C}6M4$uho`BaG*kO9tTP6?q
zs{|6j0VM1Ko*NtY3Vz<ce3H)qR=)tw_f4V)ToZi`usT^dWAlAXmwZ0!3T<!@jt1Q8
zl~4#C%yF5WM!;w2v)dO=9smK#^g;2g!`)gwFvJ;_o-7)OHu*9*`SLYcCKkwnO$p-^
zrw3u?R!Igh1b|f2Lf>`{09IFCA3Ny-B>`K1;`LYK4@erC{*BcY2ap5*7T`+q<OGle
zA=+m<ezF#~_frQj01dz%6JP-@M@t(}0VZ()@ZkXvuzIPr5AWAj7R55JkOL^BNalBb
zbR;Ao7(5%`2@Q8lYbQG!APp3Nd~}onWZ-O51OYNaM-;#T9ng76Kmj<_B~k!MAp?Nr
z!-Hjj4I>y<R9J-!lmrj8MxxgOkQ80jXN6!GhGO_=NDu)96naZYN80p-W7vjnNQIn$
z0ltSXbE9&QG!{<yhI+V%FC;w`fB_}}PgJpmdj?5*@&Pr-hl;p}yORM0umBLy3rd)V
zj97`5*oGPKhj&nf89|7Jc!{7GiarH}%JCN~XosShimLcT0T2fN?4S`_u!^!ci{zs{
zJ5X>>K#RJ#i{a%4y!eZ{Xp1}G1;AL0qNr<EfQxr$jLf)*8{kYj5CzRRjd~aW@L**{
zK#kfchBt^tDYuN<7><S&D*~__b!c$nn2y+FjpV2t%;AaTrjGI$M<i!L5=AsUFpv5;
zL*$qPg<*F<u#W<XKK+OTYlM$7FpvtFJOwEL4e*Z$83PMBk-Gzp>oA9dh>8@skqp+4
za+m-H1dl$*k^5+oNw5p|0+3Hfk|fEF%OC+Mn2{{Vk{tOturPOO#3g+&1T@K#t>^~<
zhe9FAlRZh28(@=GPyvF0LbLLYL|K$RiIil(0a(RXzU5gz=O~d-*^>oOfV=ROp%89a
zsg+yFk_=!ij4%okuxVpSmSxG20&oDUpctkA4CsY`Z%L8{a1CbAml`3GCJ+JuJ6^Mt
AyZ`_I
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/templates/_comment.html
@@ -0,0 +1,5 @@
+<?python from genshi import HTML ?>
+<li id="comment$num">
+  <strong>${comment.username}</strong> at ${comment.time.strftime('%x %X')}
+  <blockquote>${HTML(comment.content)}</blockquote>
+</li>
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/templates/_form.html
@@ -0,0 +1,24 @@
+<form xmlns="http://www.w3.org/1999/xhtml"
+      xmlns:py="http://genshi.edgewall.org/"
+      class="comment" action="${url('/comment/%s/' % link.id)}" method="post">
+  <table summary=""><tbody><tr>
+    <th><label for="username">Your name:</label></th>
+    <td>
+      <input type="text" id="username" name="username" />
+      <span py:if="'username' in errors" class="error">${errors.username}</span>
+    </td>
+  </tr><tr>
+    <th><label for="comment">Comment:</label></th>
+    <td>
+      <textarea id="comment" name="content" rows="6" cols="50"></textarea>
+      <span py:if="'content' in errors" class="error"><br />${errors.content}</span>
+      <p class="hint">You can use HTML tags here for formatting.</p>
+    </td>
+  </tr><tr>
+    <td></td>
+    <td>
+      <input type="submit" value="Submit" />
+      <input type="submit" name="cancel" value="Cancel" />
+    </td>
+  </tr></tbody></table>
+</form>
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/templates/comment.html
@@ -0,0 +1,18 @@
+<!DOCTYPE html>
+<html xmlns="http://www.w3.org/1999/xhtml"
+      xmlns:xi="http://www.w3.org/2001/XInclude"
+      xmlns:py="http://genshi.edgewall.org/">
+  <xi:include href="layout.html" />
+  <head>
+    <title>Comment on “${link.title}”</title>
+  </head>
+  <body class="comment">
+    <h1>Comment on “${link.title}”</h1>
+    <p py:if="comment">
+      In reply to <strong>${comment.username}</strong>
+      at ${comment.time.strftime('%x %X')}:
+      <blockquote>${comment.content}</blockquote>
+    </p>
+    <xi:include href="_form.html" />
+  </body>
+</html>
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/templates/index.html
@@ -0,0 +1,28 @@
+<!DOCTYPE html>
+<html xmlns="http://www.w3.org/1999/xhtml"
+      xmlns:xi="http://www.w3.org/2001/XInclude"
+      xmlns:py="http://genshi.edgewall.org/">
+  <xi:include href="layout.html" />
+  <head>
+    <title>News</title>
+    <link rel="alternate" type="application/atom+xml" title="Geddit"
+          href="${url('/feed/')}" />
+  </head>
+  <body class="index">
+    <h1>News</h1>
+
+    <ol py:if="links" class="links">
+      <li py:for="link in links">
+        <a href="${link.url}">${link.title}</a>
+        posted by ${link.username} at ${link.time.strftime('%x %X')}
+        <div class="info">
+          <a href="${url('/info/%s/' % link.id)}">
+            ${len(link.comments)} comments
+          </a>
+        </div>
+      </li>
+    </ol>
+
+    <p><a class="action" href="${url('/submit/')}">Submit new link</a></p>
+  </body>
+</html>
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/templates/index.xml
@@ -0,0 +1,23 @@
+<?xml version="1.0" encoding="utf-8"?>
+<feed xmlns="http://www.w3.org/2005/Atom"
+      xmlns:py="http://genshi.edgewall.org/">
+
+  <title>Geddit News</title>
+  <id href="${url('/')}"/>
+  <link rel="alternate" href="${url('/')}" type="text/html"/>
+  <link rel="self" href="${url('/feed/')}" type="application/atom+xml"/>
+  <updated>${links[0].time.isoformat()}</updated>
+
+  <entry py:for="link in reversed(links)">
+    <title>${link.url}</title>
+    <link rel="alternate" href="${link.url}" type="text/html"/>
+    <link rel="via" href="${url('/info/%s/' % link.id)}" type="text/html"/>
+    <id>${url('/info/%s/' % link.id)}</id>
+    <author>
+      <name>${link.username}</name>
+    </author>
+    <updated>${link.time.isoformat()}</updated>
+    <summary>${link.title}</summary>
+  </entry>
+
+</feed>
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/templates/info.html
@@ -0,0 +1,62 @@
+<!DOCTYPE html>
+<html xmlns="http://www.w3.org/1999/xhtml"
+      xmlns:xi="http://www.w3.org/2001/XInclude"
+      xmlns:py="http://genshi.edgewall.org/">
+  <xi:include href="layout.html" />
+  <head>
+    <title>${link.title}</title>
+    <link rel="alternate" type="application/atom+xml" title="Geddit: ${link.title}"
+          href="${url('/feed/%s/' % link.id)}" />
+    <script type="text/javascript">
+      function loadCommentForm(a) {
+        $.get("${url('/comment/%s/' % link.id)}", {}, function(html) {
+          var form = a.hide().parent().after(html).next();
+          function closeForm() {
+            form.slideUp("fast", function() { a.fadeIn(); form.remove() });
+            return false;
+          }
+          function initForm() {
+            form.find("input[@name='cancel']").click(closeForm);
+            form.submit(function() {
+              var data = form.find("input[@type='text'], textarea").serialize();
+              $.post("${url('/comment/%s/' % link.id)}", data, function(html) {
+                var elem = $(html).get(0);
+                if (/form/i.test(elem.tagName)) {
+                  form.after(elem).remove();
+                  form = $(elem);
+                  initForm();
+                } else {
+                  if ($("ul.comments").length == 0) {
+                    a.parent().before('<ul class="comments"></ul>');
+                  }
+                  $("ul.comments").append($(elem));
+                  closeForm();
+                }
+              });
+              return false;
+            });
+          }
+          initForm();
+        });
+      }
+      $(document).ready(function() {
+        $("a.action").click(function() {
+          loadCommentForm($(this));
+          return false;
+        });
+      });
+    </script>
+  </head>
+  <body class="info">
+    <h1>${link.title}</h1>
+    <a href="${link.url}">${link.url}</a><br />
+    posted by ${link.username} at ${link.time.strftime('%x %X')}<br />
+
+    <ul py:if="link.comments" class="comments">
+      <xi:include href="_comment.html"
+          py:for="num, comment in enumerate(link.comments)" />
+    </ul>
+
+    <p><a class="action" href="${url('/comment/%s/' % link.id)}">Add comment</a></p>
+  </body>
+</html>
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/templates/info.xml
@@ -0,0 +1,28 @@
+<?xml version="1.0" encoding="utf-8"?>
+<feed xmlns="http://www.w3.org/2005/Atom"
+      xmlns:py="http://genshi.edgewall.org/">
+
+  <title>Geddit: ${link.title}</title>
+  <id href="${url('/info/%s/' % link.id)}"/>
+  <link rel="alternate" href="${url('/info/%s/' % link.id)}" type="text/html"/>
+  <link rel="self" href="${url('/feed/%s/' % link.id)}" type="application/atom+xml"/>
+  <updated py:with="time=link.comments and link.comments[-1].time or link.time">
+    ${time.isoformat()}
+  </updated>
+
+  <?python from genshi import HTML ?>
+  <entry py:for="idx, comment in enumerate(reversed(link.comments))">
+    <title>Comment ${len(link.comments) - idx} on “${link.title}”</title>
+    <link rel="alternate" href="${url('/info/%s/' % link.id)}#comment${idx}"
+          type="text/html"/>
+    <id>${url('/info/%s/' % link.id)}#comment${idx}</id>
+    <author>
+      <name>${comment.username}</name>
+    </author>
+    <updated>${comment.time.isoformat()}</updated>
+    <content type="xhtml"><div xmlns="http://www.w3.org/1999/xhtml">
+      ${HTML(comment.content)}
+    </div></content>
+  </entry>
+
+</feed>
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/templates/layout.html
@@ -0,0 +1,31 @@
+<!DOCTYPE html>
+<html xmlns="http://www.w3.org/1999/xhtml"
+      xmlns:py="http://genshi.edgewall.org/" py:strip="">
+
+  <py:match path="head" once="true">
+    <head py:attrs="select('@*')">
+      <title py:with="title = list(select('title/text()'))">
+        Geddit<py:if test="title">: ${title}</py:if>
+      </title>
+      <link rel="stylesheet" href="${url('/media/layout.css')}" type="text/css" />
+      <script type="text/javascript" src="${url('/media/jquery.js')}"></script>
+      ${select('*[local-name()!="title"]')}
+    </head>
+  </py:match>
+
+  <py:match path="body" once="true">
+    <body py:attrs="select('@*')"><div id="wrap">
+      <div id="header">
+        <a href="/"><img src="${url('/media/logo.gif')}" width="201" height="79" alt="geddit?" /></a>
+      </div>
+      <div id="content">
+        ${select('*|text()')}
+      </div>
+      <div id="footer">
+        <hr />
+        <p class="legalese">© 2007 Edgewall Software</p>
+      </div>
+    </div></body>
+  </py:match>
+
+</html>
new file mode 100644
--- /dev/null
+++ b/examples/tutorial/geddit/templates/submit.html
@@ -0,0 +1,41 @@
+<!DOCTYPE html>
+<html xmlns="http://www.w3.org/1999/xhtml"
+      xmlns:xi="http://www.w3.org/2001/XInclude"
+      xmlns:py="http://genshi.edgewall.org/">
+  <xi:include href="layout.html" />
+  <head>
+    <title>Submit new link</title>
+  </head>
+  <body class="submit">
+    <h1>Submit new link</h1>
+
+    <form action="" method="post">
+      <table summary=""><tbody><tr>
+        <th><label for="username">Your name:</label></th>
+        <td>
+          <input type="text" id="username" name="username" />
+          <span py:if="'username' in errors" class="error">${errors.username}</span>
+        </td>
+      </tr><tr>
+        <th><label for="url">Link URL:</label></th>
+        <td>
+          <input type="text" id="url" name="url" />
+          <span py:if="'url' in errors" class="error">${errors.url}</span>
+        </td>
+      </tr><tr>
+        <th><label for="title">Title:</label></th>
+        <td>
+          <input type="text" name="title" />
+          <span py:if="'title' in errors" class="error">${errors.title}</span>
+        </td>
+      </tr><tr>
+        <td></td>
+        <td>
+          <input type="submit" value="Submit" />
+          <input type="submit" name="cancel" value="Cancel" />
+        </td>
+      </tr></tbody></table>
+    </form>
+
+  </body>
+</html>
--- a/genshi/__init__.py
+++ b/genshi/__init__.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006 Edgewall Software
+# Copyright (C) 2006-2007 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
--- a/genshi/_speedups.c
+++ b/genshi/_speedups.c
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2006 Edgewall Software
+ * Copyright (C) 2006-2008 Edgewall Software
  * All rights reserved.
  *
  * This software is licensed as described in the file COPYING, which
@@ -87,6 +87,7 @@
 
     out = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, len);
     if (out == NULL) {
+        Py_DECREF((PyObject *) in);
         return NULL;
     }
 
@@ -130,6 +131,8 @@
         inp++;
     }
 
+    Py_DECREF((PyObject *) in);
+
     args = PyTuple_New(1);
     if (args == NULL) {
         Py_DECREF((PyObject *) out);
@@ -141,46 +144,6 @@
     return ret;
 }
 
-static PyObject *
-Markup_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
-{
-    PyObject *self, *text, *tmp, *args2;
-    int nargs, i;
-
-    nargs = PyTuple_GET_SIZE(args);
-    if (nargs < 2) {
-        return PyUnicode_Type.tp_new(type, args, NULL);
-    }
-
-    text = PyTuple_GET_ITEM(args, 0);
-    args2 = PyTuple_New(nargs - 1);
-    if (args2 == NULL) {
-        return NULL;
-    }
-    for (i = 1; i < nargs; i++) {
-        tmp = escape(PyTuple_GET_ITEM(args, i), 1);
-        if (tmp == NULL) {
-            Py_DECREF(args2);
-            return NULL;
-        }
-        PyTuple_SET_ITEM(args2, i - 1, tmp);
-    }
-    tmp = PyUnicode_Format(text, args2);
-    Py_DECREF(args2);
-    if (tmp == NULL) {
-        return NULL;
-    }
-    args = PyTuple_New(1);
-    if (args == NULL) {
-        Py_DECREF(tmp);
-        return NULL;
-    }
-    PyTuple_SET_ITEM(args, 0, tmp);
-    self = PyUnicode_Type.tp_new(type, args, NULL);
-    Py_DECREF(args);
-    return self;
-}
-
 PyDoc_STRVAR(escape__doc__,
 "Create a Markup instance from a string and escape special characters\n\
 it may contain (<, >, & and \").\n\
@@ -242,7 +205,7 @@
 Markup_join(PyObject *self, PyObject *args, PyObject *kwds)
 {
     static char *kwlist[] = {"seq", "escape_quotes", 0};
-    PyObject *seq = NULL, *seq2, *tmp;
+    PyObject *seq = NULL, *seq2, *tmp, *tmp2;
     char quotes = 1;
     int n, i;
 
@@ -266,12 +229,13 @@
             Py_DECREF(seq2);
             return NULL;
         }
-        tmp = escape(tmp, quotes);
-        if (tmp == NULL) {
+        tmp2 = escape(tmp, quotes);
+        if (tmp2 == NULL) {
             Py_DECREF(seq2);
             return NULL;
         }
-        PyTuple_SET_ITEM(seq2, i, tmp);
+        PyTuple_SET_ITEM(seq2, i, tmp2);
+        Py_DECREF(tmp);
     }
     tmp = PyUnicode_Join(self, seq2);
     Py_DECREF(seq2);
@@ -303,11 +267,9 @@
             return NULL;
         tmp2 = PyUnicode_Concat(tmp, other);
     }
-    if (tmp2 == NULL) {
-        Py_DECREF(tmp);
+    Py_DECREF(tmp);
+    if (tmp2 == NULL)
         return NULL;
-    }
-    Py_DECREF(tmp);
     args = PyTuple_New(1);
     if (args == NULL) {
         Py_DECREF(tmp2);
@@ -323,9 +285,38 @@
 Markup_mod(PyObject *self, PyObject *args)
 {
     PyObject *tmp, *tmp2, *ret, *args2;
-    int i, nargs;
+    int i, nargs = 0;
+    PyObject *kwds = NULL;
 
-    if (PyTuple_Check(args)) {
+    if (PyDict_Check(args)) {
+        kwds = args;
+    }
+    if (kwds && PyDict_Size(kwds)) {
+        PyObject *kwcopy, *key, *value;
+        Py_ssize_t pos = 0;
+
+        kwcopy = PyDict_Copy( kwds );
+        if (kwcopy == NULL) {
+            return NULL;
+        }
+        while (PyDict_Next(kwcopy, &pos, &key, &value)) {
+            tmp = escape(value, 1);
+            if (tmp == NULL) {
+                Py_DECREF(kwcopy);
+                return NULL;
+            }
+            if (PyDict_SetItem(kwcopy, key, tmp) < 0) {
+                Py_DECREF(tmp);
+                Py_DECREF(kwcopy);
+                return NULL;
+            }
+        }
+        tmp = PyUnicode_Format(self, kwcopy);
+        Py_DECREF(kwcopy);
+        if (tmp == NULL) {
+            return NULL;
+        }
+    } else if (PyTuple_Check(args)) {
         nargs = PyTuple_GET_SIZE(args);
         args2 = PyTuple_New(nargs);
         if (args2 == NULL) {
@@ -380,6 +371,7 @@
         if (unicode == NULL) return NULL;
         result = PyNumber_Multiply(unicode, self);
     }
+    Py_DECREF(unicode);
 
     if (result == NULL) return NULL;
     args = PyTuple_New(1);
@@ -402,14 +394,19 @@
     format = PyString_FromString("<Markup %r>");
     if (format == NULL) return NULL;
     result = PyObject_Unicode(self);
-    if (result == NULL) return NULL;
+    if (result == NULL) {
+        Py_DECREF(format);
+        return NULL;
+    }
     args = PyTuple_New(1);
     if (args == NULL) {
+        Py_DECREF(format);
         Py_DECREF(result);
         return NULL;
     }
     PyTuple_SET_ITEM(args, 0, result);
     result = PyString_Format(format, args);
+    Py_DECREF(format);
     Py_DECREF(args);
     return result;
 }
@@ -581,7 +578,7 @@
     
     0,          /*tp_init*/
     0,          /*tp_alloc  will be set to PyType_GenericAlloc in module init*/
-    Markup_new, /*tp_new*/
+    0,          /*tp_new*/
     0,          /*tp_free  Low-level free-memory routine */
     0,          /*tp_is_gc For PyObject_IS_GC */
     0,          /*tp_bases*/
--- a/genshi/core.py
+++ b/genshi/core.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -13,6 +13,7 @@
 
 """Core classes for markup processing."""
 
+from itertools import chain
 import operator
 
 from genshi.util import plaintext, stripentities, striptags
@@ -51,7 +52,7 @@
     returns the complete generated text at once. Both accept various parameters
     that impact the way the stream is serialized.
     """
-    __slots__ = ['events']
+    __slots__ = ['events', 'serializer']
 
     START = StreamEventKind('START') #: a start tag
     END = StreamEventKind('END') #: an end tag
@@ -65,12 +66,17 @@
     PI = StreamEventKind('PI') #: processing instruction
     COMMENT = StreamEventKind('COMMENT') #: comment
 
-    def __init__(self, events):
+    def __init__(self, events, serializer=None):
         """Initialize the stream with a sequence of markup events.
         
         :param events: a sequence or iterable providing the events
+        :param serializer: the default serialization method to use for this
+                           stream
+
+        :note: Changed in 0.5: added the `serializer` argument
         """
         self.events = events #: The underlying iterable producing the events
+        self.serializer = serializer #: The default serializion method
 
     def __iter__(self):
         return iter(self.events)
@@ -119,7 +125,7 @@
         :return: the filtered stream
         :rtype: `Stream`
         """
-        return Stream(_ensure(function(self)))
+        return Stream(_ensure(function(self)), serializer=self.serializer)
 
     def filter(self, *filters):
         """Apply filters to the stream.
@@ -143,7 +149,7 @@
         """
         return reduce(operator.or_, (self,) + filters)
 
-    def render(self, method='xml', encoding='utf-8', **kwargs):
+    def render(self, method=None, encoding='utf-8', out=None, **kwargs):
         """Return a string representation of the stream.
         
         Any additional keyword arguments are passed to the serializer, and thus
@@ -151,21 +157,52 @@
         
         :param method: determines how the stream is serialized; can be either
                        "xml", "xhtml", "html", "text", or a custom serializer
-                       class
+                       class; if `None`, the default serialization method of
+                       the stream is used
         :param encoding: how the output string should be encoded; if set to
                          `None`, this method returns a `unicode` object
-        :return: a `str` or `unicode` object
+        :param out: a file-like object that the output should be written to
+                    instead of being returned as one big string; note that if
+                    this is a file or socket (or similar), the `encoding` must
+                    not be `None` (that is, the output must be encoded)
+        :return: a `str` or `unicode` object (depending on the `encoding`
+                 parameter), or `None` if the `out` parameter is provided
         :rtype: `basestring`
+        
         :see: XMLSerializer, XHTMLSerializer, HTMLSerializer, TextSerializer
+        :note: Changed in 0.5: added the `out` parameter
         """
         from genshi.output import encode
+        if method is None:
+            method = self.serializer or 'xml'
         generator = self.serialize(method=method, **kwargs)
-        return encode(generator, method=method, encoding=encoding)
+        return encode(generator, method=method, encoding=encoding, out=out)
 
     def select(self, path, namespaces=None, variables=None):
         """Return a new stream that contains the events matching the given
         XPath expression.
         
+        >>> from genshi import HTML
+        >>> stream = HTML('<doc><elem>foo</elem><elem>bar</elem></doc>')
+        >>> print stream.select('elem')
+        <elem>foo</elem><elem>bar</elem>
+        >>> print stream.select('elem/text()')
+        foobar
+        
+        Note that the outermost element of the stream becomes the *context
+        node* for the XPath test. That means that the expression "doc" would
+        not match anything in the example above, because it only tests against
+        child elements of the outermost element:
+        
+        >>> print stream.select('doc')
+        <BLANKLINE>
+        
+        You can use the "." expression to match the context node itself
+        (although that usually makes little sense):
+        
+        >>> print stream.select('.')
+        <doc><elem>foo</elem><elem>bar</elem></doc>
+        
         :param path: a string containing the XPath expression
         :param namespaces: mapping of namespace prefixes used in the path
         :param variables: mapping of variable names to values
@@ -190,13 +227,16 @@
         
         :param method: determines how the stream is serialized; can be either
                        "xml", "xhtml", "html", "text", or a custom serializer
-                       class
+                       class; if `None`, the default serialization method of
+                       the stream is used
         :return: an iterator over the serialization results (`Markup` or
                  `unicode` objects, depending on the serialization method)
         :rtype: ``iterator``
         :see: XMLSerializer, XHTMLSerializer, HTMLSerializer, TextSerializer
         """
         from genshi.output import get_serializer
+        if method is None:
+            method = self.serializer or 'xml'
         return get_serializer(method, **kwargs)(_ensure(self))
 
     def __str__(self):
@@ -220,12 +260,24 @@
 
 def _ensure(stream):
     """Ensure that every item on the stream is actually a markup event."""
-    for event in stream:
-        if type(event) is not tuple:
+    stream = iter(stream)
+    event = stream.next()
+
+    # Check whether the iterable is a real markup event stream by examining the
+    # first item it yields; if it's not we'll need to do some conversion
+    if type(event) is not tuple or len(event) != 3:
+        for event in chain([event], stream):
             if hasattr(event, 'totuple'):
                 event = event.totuple()
             else:
                 event = TEXT, unicode(event), (None, -1, -1)
+            yield event
+        return
+
+    # This looks like a markup event stream, so we'll just pass it through
+    # unchanged
+    yield event
+    for event in stream:
         yield event
 
 
@@ -295,6 +347,12 @@
                 return True
 
     def __getslice__(self, i, j):
+        """Return a slice of the attributes list.
+        
+        >>> attrs = Attrs([('href', '#'), ('title', 'Foo')])
+        >>> attrs[1:]
+        Attrs([('title', 'Foo')])
+        """
         return Attrs(tuple.__getslice__(self, i, j))
 
     def __or__(self, attrs):
@@ -361,11 +419,6 @@
     """
     __slots__ = []
 
-    def __new__(cls, text='', *args):
-        if args:
-            text %= tuple(map(escape, args))
-        return unicode.__new__(cls, text)
-
     def __add__(self, other):
         return Markup(unicode(self) + unicode(escape(other)))
 
@@ -373,9 +426,13 @@
         return Markup(unicode(escape(other)) + unicode(self))
 
     def __mod__(self, args):
-        if not isinstance(args, (list, tuple)):
-            args = [args]
-        return Markup(unicode.__mod__(self, tuple(map(escape, args))))
+        if isinstance(args, dict):
+            args = dict(zip(args.keys(), map(escape, args.values())))
+        elif isinstance(args, (list, tuple)):
+            args = tuple(map(escape, args))
+        else:
+            args = escape(args)
+        return Markup(unicode.__mod__(self, args))
 
     def __mul__(self, num):
         return Markup(unicode(self) * num)
@@ -548,7 +605,7 @@
     def __new__(cls, uri):
         if type(uri) is cls:
             return uri
-        return object.__new__(cls, uri)
+        return object.__new__(cls)
 
     def __getnewargs__(self):
         return (self.uri,)
@@ -595,9 +652,9 @@
     """A qualified element or attribute name.
     
     The unicode value of instances of this class contains the qualified name of
-    the element or attribute, in the form ``{namespace}localname``. The namespace
-    URI can be obtained through the additional `namespace` attribute, while the
-    local name can be accessed through the `localname` attribute.
+    the element or attribute, in the form ``{namespace-uri}local-name``. The
+    namespace URI can be obtained through the additional `namespace` attribute,
+    while the local name can be accessed through the `localname` attribute.
     
     >>> qname = QName('foo')
     >>> qname
@@ -617,10 +674,16 @@
     __slots__ = ['namespace', 'localname']
 
     def __new__(cls, qname):
+        """Create the `QName` instance.
+        
+        :param qname: the qualified name as a string of the form
+                      ``{namespace-uri}local-name``, where the leading curly
+                      brace is optional
+        """
         if type(qname) is cls:
             return qname
 
-        parts = qname.split(u'}', 1)
+        parts = qname.lstrip(u'{').split(u'}', 1)
         if len(parts) > 1:
             self = unicode.__new__(cls, u'{%s' % qname)
             self.namespace, self.localname = map(unicode, parts)
--- a/genshi/filters/html.py
+++ b/genshi/filters/html.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -14,13 +14,14 @@
 """Implementation of a number of stream filters."""
 
 try:
-    frozenset
+    set
 except NameError:
     from sets import ImmutableSet as frozenset
+    from sets import Set as set
 import re
 
 from genshi.core import Attrs, QName, stripentities
-from genshi.core import END, START, TEXT
+from genshi.core import END, START, TEXT, COMMENT
 
 __all__ = ['HTMLFormFiller', 'HTMLSanitizer']
 __docformat__ = 'restructuredtext en'
@@ -69,7 +70,9 @@
         """
         in_form = in_select = in_option = in_textarea = False
         select_value = option_value = textarea_value = None
-        option_start = option_text = None
+        option_start = None
+        option_text = []
+        no_option_value = False
 
         for kind, data, pos in stream:
 
@@ -94,13 +97,13 @@
                                 checked = False
                                 if isinstance(value, (list, tuple)):
                                     if declval:
-                                        checked = declval in [str(v) for v
+                                        checked = declval in [unicode(v) for v
                                                               in value]
                                     else:
                                         checked = bool(filter(None, value))
                                 else:
                                     if declval:
-                                        checked = declval == str(value)
+                                        checked = declval == unicode(value)
                                     elif type == 'checkbox':
                                         checked = bool(value)
                                 if checked:
@@ -130,15 +133,18 @@
                     elif in_select and tagname == 'option':
                         option_start = kind, data, pos
                         option_value = attrs.get('value')
+                        if option_value is None:
+                            no_option_value = True
+                            option_value = ''
                         in_option = True
                         continue
                 yield kind, (tag, attrs), pos
 
             elif in_form and kind is TEXT:
                 if in_select and in_option:
-                    if option_value is None:
-                        option_value = data
-                    option_text = kind, data, pos
+                    if no_option_value:
+                        option_value += data
+                    option_text.append((kind, data, pos))
                     continue
                 elif in_textarea:
                     continue
@@ -153,10 +159,10 @@
                     select_value = None
                 elif in_select and tagname == 'option':
                     if isinstance(select_value, (tuple, list)):
-                        selected = option_value in [str(v) for v
+                        selected = option_value in [unicode(v) for v
                                                     in select_value]
                     else:
-                        selected = option_value == str(select_value)
+                        selected = option_value == unicode(select_value)
                     okind, (tag, attrs), opos = option_start
                     if selected:
                         attrs |= [(QName('selected'), 'selected')]
@@ -164,9 +170,12 @@
                         attrs -= 'selected'
                     yield okind, (tag, attrs), opos
                     if option_text:
-                        yield option_text
+                        for event in option_text:
+                            yield event
                     in_option = False
-                    option_start = option_text = option_value = None
+                    no_option_value = False
+                    option_start = option_value = None
+                    option_text = []
                 elif tagname == 'textarea':
                     if textarea_value:
                         yield TEXT, unicode(textarea_value), pos
@@ -206,6 +215,9 @@
     well as a lot of other things. However, the style tag is still excluded by
     default because it is very hard for such sanitizing to be completely safe,
     especially considering how much error recovery current web browsers perform.
+    
+    :warn: Note that this special processing of CSS is currently only applied to
+           style attributes, **not** style elements.
     """
 
     SAFE_TAGS = frozenset(['a', 'abbr', 'acronym', 'address', 'area', 'b',
@@ -247,9 +259,13 @@
         :param uri_attrs: a set of names of attributes that contain URIs
         """
         self.safe_tags = safe_tags
+        "The set of tag names that are considered safe."
         self.safe_attrs = safe_attrs
+        "The set of attribute names that are considered safe."
         self.uri_attrs = uri_attrs
+        "The set of names of attributes that may contain URIs."
         self.safe_schemes = safe_schemes
+        "The set of URI schemes that are considered safe."
 
     def __call__(self, stream):
         """Apply the filter to the given stream.
@@ -258,12 +274,6 @@
         """
         waiting_for = None
 
-        def _get_scheme(href):
-            if ':' not in href:
-                return None
-            chars = [char for char in href.split(':', 1)[0] if char.isalnum()]
-            return ''.join(chars).lower()
-
         for kind, data, pos in stream:
             if kind is START:
                 if waiting_for:
@@ -280,24 +290,11 @@
                         continue
                     elif attr in self.uri_attrs:
                         # Don't allow URI schemes such as "javascript:"
-                        if _get_scheme(value) not in self.safe_schemes:
+                        if not self.is_safe_uri(value):
                             continue
                     elif attr == 'style':
                         # Remove dangerous CSS declarations from inline styles
-                        decls = []
-                        value = self._strip_css_comments(
-                            self._replace_unicode_escapes(value)
-                        )
-                        for decl in filter(None, value.split(';')):
-                            is_evil = False
-                            if 'expression' in decl:
-                                is_evil = True
-                            for m in re.finditer(r'url\s*\(([^)]+)', decl):
-                                if _get_scheme(m.group(1)) not in self.safe_schemes:
-                                    is_evil = True
-                                    break
-                            if not is_evil:
-                                decls.append(decl.strip())
+                        decls = self.sanitize_css(value)
                         if not decls:
                             continue
                         value = '; '.join(decls)
@@ -313,10 +310,79 @@
                 else:
                     yield kind, data, pos
 
-            else:
+            elif kind is not COMMENT:
                 if not waiting_for:
                     yield kind, data, pos
 
+    def is_safe_uri(self, uri):
+        """Determine whether the given URI is to be considered safe for
+        inclusion in the output.
+        
+        The default implementation checks whether the scheme of the URI is in
+        the set of allowed URIs (`safe_schemes`).
+        
+        >>> sanitizer = HTMLSanitizer()
+        >>> sanitizer.is_safe_uri('http://example.org/')
+        True
+        >>> sanitizer.is_safe_uri('javascript:alert(document.cookie)')
+        False
+        
+        :param uri: the URI to check
+        :return: `True` if the URI can be considered safe, `False` otherwise
+        :rtype: `bool`
+        :since: version 0.4.3
+        """
+        if ':' not in uri:
+            return True # This is a relative URI
+        chars = [char for char in uri.split(':', 1)[0] if char.isalnum()]
+        return ''.join(chars).lower() in self.safe_schemes
+
+    def sanitize_css(self, text):
+        """Remove potentially dangerous property declarations from CSS code.
+        
+        In particular, properties using the CSS ``url()`` function with a scheme
+        that is not considered safe are removed:
+        
+        >>> sanitizer = HTMLSanitizer()
+        >>> sanitizer.sanitize_css(u'''
+        ...   background: url(javascript:alert("foo"));
+        ...   color: #000;
+        ... ''')
+        [u'color: #000']
+        
+        Also, the proprietary Internet Explorer function ``expression()`` is
+        always stripped:
+        
+        >>> sanitizer.sanitize_css(u'''
+        ...   background: #fff;
+        ...   color: #000;
+        ...   width: e/**/xpression(alert("foo"));
+        ... ''')
+        [u'background: #fff', u'color: #000']
+        
+        :param text: the CSS text; this is expected to be `unicode` and to not
+                     contain any character or numeric references
+        :return: a list of declarations that are considered safe
+        :rtype: `list`
+        :since: version 0.4.3
+        """
+        decls = []
+        text = self._strip_css_comments(self._replace_unicode_escapes(text))
+        for decl in filter(None, text.split(';')):
+            decl = decl.strip()
+            if not decl:
+                continue
+            is_evil = False
+            if 'expression' in decl:
+                is_evil = True
+            for match in re.finditer(r'url\s*\(([^)]+)', decl):
+                if not self.is_safe_uri(match.group(1)):
+                    is_evil = True
+                    break
+            if not is_evil:
+                decls.append(decl.strip())
+        return decls
+
     _NORMALIZE_NEWLINES = re.compile(r'\r\n').sub
     _UNICODE_ESCAPE = re.compile(r'\\([0-9a-fA-F]{1,6})\s?').sub
 
--- a/genshi/filters/i18n.py
+++ b/genshi/filters/i18n.py
@@ -11,7 +11,10 @@
 # individuals. For the exact contribution history, see the revision
 # history and logs, available at http://genshi.edgewall.org/log/.
 
-"""Utilities for internationalization and localization of templates."""
+"""Utilities for internationalization and localization of templates.
+
+:since: version 0.4
+"""
 
 from compiler import ast
 try:
@@ -93,17 +96,21 @@
                                'summary', 'title'])
 
     def __init__(self, translate=gettext, ignore_tags=IGNORE_TAGS,
-                 include_attrs=INCLUDE_ATTRS):
+                 include_attrs=INCLUDE_ATTRS, extract_text=True):
         """Initialize the translator.
         
         :param translate: the translation function, for example ``gettext`` or
                           ``ugettext``.
         :param ignore_tags: a set of tag names that should not be localized
         :param include_attrs: a set of attribute names should be localized
+        :param extract_text: whether the content of text nodes should be
+                             extracted, or only text in explicit ``gettext``
+                             function calls
         """
         self.translate = translate
         self.ignore_tags = ignore_tags
         self.include_attrs = include_attrs
+        self.extract_text = extract_text
 
     def __call__(self, stream, ctxt=None, search_text=True, msgbuf=None):
         """Translate any localizable strings in the given stream.
@@ -124,6 +131,8 @@
         ignore_tags = self.ignore_tags
         include_attrs = self.include_attrs
         translate = self.translate
+        if not self.extract_text:
+            search_text = False
         skip = 0
         i18n_msg = I18N_NAMESPACE['msg']
         ns_prefixes = []
@@ -153,7 +162,7 @@
                 changed = False
                 for name, value in attrs:
                     newval = value
-                    if isinstance(value, basestring):
+                    if search_text and isinstance(value, basestring):
                         if name in include_attrs:
                             newval = self.translate(value)
                     else:
@@ -165,7 +174,7 @@
                         changed = True
                     new_attrs.append((name, value))
                 if changed:
-                    attrs = new_attrs
+                    attrs = Attrs(new_attrs)
 
                 if msgbuf:
                     msgbuf.append(kind, data, pos)
@@ -255,6 +264,8 @@
                (such as ``ngettext``), a single item with a tuple of strings is
                yielded, instead an item for each string argument.
         """
+        if not self.extract_text:
+            search_text = False
         skip = 0
         i18n_msg = I18N_NAMESPACE['msg']
         xml_lang = XML_NAMESPACE['lang']
@@ -276,7 +287,7 @@
                     continue
 
                 for name, value in attrs:
-                    if isinstance(value, basestring):
+                    if search_text and isinstance(value, basestring):
                         if name in self.include_attrs:
                             text = value.strip()
                             if text:
@@ -321,7 +332,10 @@
 
 
 class MessageBuffer(object):
-    """Helper class for managing localizable mixed content."""
+    """Helper class for managing localizable mixed content.
+    
+    :since: version 0.5
+    """
 
     def __init__(self, lineno=-1):
         self.lineno = lineno
@@ -385,17 +399,21 @@
     :param code: the `Code` object
     :type code: `genshi.template.eval.Code`
     :param gettext_functions: a sequence of function names
+    :since: version 0.5
     """
     def _walk(node):
         if isinstance(node, ast.CallFunc) and isinstance(node.node, ast.Name) \
                 and node.node.name in gettext_functions:
             strings = []
-            for arg in node.args:
+            def _add(arg):
                 if isinstance(arg, ast.Const) \
                         and isinstance(arg.value, basestring):
-                    strings.append(unicode(arg.value))
-                elif not isinstance(arg, ast.Keyword):
+                    strings.append(unicode(arg.value, 'utf-8'))
+                elif arg and not isinstance(arg, ast.Keyword):
                     strings.append(None)
+            [_add(arg) for arg in node.args]
+            _add(node.star_args)
+            _add(node.dstar_args)
             if len(strings) == 1:
                 strings = strings[0]
             else:
@@ -421,6 +439,8 @@
     
     >>> parse_msg("[1:] Bilder pro Seite anzeigen.")
     [(1, ''), (0, ' Bilder pro Seite anzeigen.')]
+    
+    :since: version 0.5
     """
     parts = []
     stack = [0]
@@ -464,10 +484,15 @@
         template_class = getattr(__import__(module, {}, {}, [clsname]), clsname)
     encoding = options.get('encoding', None)
 
+    extract_text = options.get('extract_text', True)
+    if isinstance(extract_text, basestring):
+        extract_text = extract_text.lower() in ('1', 'on', 'yes', 'true')
+
     ignore_tags = options.get('ignore_tags', Translator.IGNORE_TAGS)
     if isinstance(ignore_tags, basestring):
         ignore_tags = ignore_tags.split()
     ignore_tags = [QName(tag) for tag in ignore_tags]
+
     include_attrs = options.get('include_attrs', Translator.INCLUDE_ATTRS)
     if isinstance(include_attrs, basestring):
         include_attrs = include_attrs.split()
@@ -475,7 +500,7 @@
 
     tmpl = template_class(fileobj, filename=getattr(fileobj, 'name', None),
                           encoding=encoding)
-    translator = Translator(None, ignore_tags, include_attrs)
+    translator = Translator(None, ignore_tags, include_attrs, extract_text)
     for lineno, func, message in translator.extract(tmpl.stream,
                                                     gettext_functions=keywords):
         yield lineno, func, message, []
--- a/genshi/filters/tests/__init__.py
+++ b/genshi/filters/tests/__init__.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2007 Edgewall Software
+# Copyright (C) 2007-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -19,7 +19,8 @@
     suite = unittest.TestSuite()
     suite.addTest(html.suite())
     suite.addTest(i18n.suite())
-    suite.addTest(transform.suite())
+    if hasattr(doctest, 'NORMALIZE_WHITESPACE'):
+        suite.addTest(transform.suite())
     return suite
 
 if __name__ == '__main__':
--- a/genshi/filters/tests/html.py
+++ b/genshi/filters/tests/html.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -12,11 +12,15 @@
 # history and logs, available at http://genshi.edgewall.org/log/.
 
 import doctest
+try:
+    set
+except NameError:
+    from sets import Set as set
 import unittest
 
 from genshi.input import HTML, ParseError
 from genshi.filters.html import HTMLFormFiller, HTMLSanitizer
-
+from genshi.template import MarkupTemplate
 
 class HTMLFormFillerTestCase(unittest.TestCase):
 
@@ -270,6 +274,42 @@
           </select>
         </p></form>""", unicode(html))
 
+    def test_fill_option_segmented_text(self):
+        html = MarkupTemplate("""<form>
+          <select name="foo">
+            <option value="1">foo $x</option>
+          </select>
+        </form>""").generate(x=1) | HTMLFormFiller(data={'foo': '1'})
+        self.assertEquals("""<form>
+          <select name="foo">
+            <option value="1" selected="selected">foo 1</option>
+          </select>
+        </form>""", unicode(html))
+
+    def test_fill_option_segmented_text_no_value(self):
+        html = MarkupTemplate("""<form>
+          <select name="foo">
+            <option>foo $x bar</option>
+          </select>
+        </form>""").generate(x=1) | HTMLFormFiller(data={'foo': 'foo 1 bar'})
+        self.assertEquals("""<form>
+          <select name="foo">
+            <option selected="selected">foo 1 bar</option>
+          </select>
+        </form>""", unicode(html))
+
+    def test_fill_option_unicode_value(self):
+        html = HTML(u"""<form>
+          <select name="foo">
+            <option value="&ouml;">foo</option>
+          </select>
+        </form>""") | HTMLFormFiller(data={'foo': u'ö'})
+        self.assertEquals(u"""<form>
+          <select name="foo">
+            <option value="ö" selected="selected">foo</option>
+          </select>
+        </form>""", unicode(html))
+
 
 class HTMLSanitizerTestCase(unittest.TestCase):
 
@@ -318,6 +358,10 @@
         html = HTML('<div onclick=\'alert("foo")\' />')
         self.assertEquals(u'<div/>', unicode(html | HTMLSanitizer()))
 
+    def test_sanitize_remove_comments(self):
+        html = HTML('''<div><!-- conditional comment crap --></div>''')
+        self.assertEquals(u'<div/>', unicode(html | HTMLSanitizer()))
+
     def test_sanitize_remove_style_scripts(self):
         sanitizer = HTMLSanitizer(safe_attrs=HTMLSanitizer.SAFE_ATTRS | set(['style']))
         # Inline style with url() using javascript: scheme
--- a/genshi/filters/tests/i18n.py
+++ b/genshi/filters/tests/i18n.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2007 Edgewall Software
+# Copyright (C) 2007-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -15,12 +15,37 @@
 from StringIO import StringIO
 import unittest
 
+from genshi.core import Attrs
 from genshi.template import MarkupTemplate
 from genshi.filters.i18n import Translator, extract
+from genshi.input import HTML
 
 
 class TranslatorTestCase(unittest.TestCase):
 
+    def test_translate_included_attribute_text(self):
+        """
+        Verify that translated attributes end up in a proper `Attrs` instance.
+        """
+        html = HTML("""<html>
+          <span title="Foo"></span>
+        </html>""")
+        translator = Translator(lambda s: u"Voh")
+        stream = list(html.filter(translator))
+        kind, data, pos = stream[2]
+        assert isinstance(data[1], Attrs)
+
+    def test_extract_without_text(self):
+        tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/">
+          <p title="Bar">Foo</p>
+          ${ngettext("Singular", "Plural", num)}
+        </html>""")
+        translator = Translator(extract_text=False)
+        messages = list(translator.extract(tmpl.stream))
+        self.assertEqual(1, len(messages))
+        self.assertEqual((3, 'ngettext', (u'Singular', u'Plural', None)),
+                         messages[0])
+
     def test_extract_plural_form(self):
         tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/">
           ${ngettext("Singular", "Plural", num)}
@@ -31,6 +56,24 @@
         self.assertEqual((2, 'ngettext', (u'Singular', u'Plural', None)),
                          messages[0])
 
+    def test_extract_funky_plural_form(self):
+        tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/">
+          ${ngettext(len(items), *widget.display_names)}
+        </html>""")
+        translator = Translator()
+        messages = list(translator.extract(tmpl.stream))
+        self.assertEqual(1, len(messages))
+        self.assertEqual((2, 'ngettext', (None, None)), messages[0])
+
+    def test_extract_gettext_with_unicode_string(self):
+        tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/">
+          ${gettext("Grüße")}
+        </html>""")
+        translator = Translator()
+        messages = list(translator.extract(tmpl.stream))
+        self.assertEqual(1, len(messages))
+        self.assertEqual((2, 'gettext', u'Gr\xfc\xdfe'), messages[0])
+
     def test_extract_included_attribute_text(self):
         tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/">
           <span title="Foo"></span>
@@ -268,6 +311,18 @@
                              []),
         ], results)
 
+    def test_extraction_without_text(self):
+        buf = StringIO("""<html xmlns:py="http://genshi.edgewall.org/">
+          <p title="Bar">Foo</p>
+          ${ngettext("Singular", "Plural", num)}
+        </html>""")
+        results = list(extract(buf, ['_', 'ngettext'], [], {
+            'extract_text': 'no'
+        }))
+        self.assertEqual([
+            (3, 'ngettext', (u'Singular', u'Plural', None), []),
+        ], results)
+
     def test_text_template_extraction(self):
         buf = StringIO("""${_("Dear %(name)s") % {'name': name}},
         
@@ -332,7 +387,7 @@
 
 def suite():
     suite = unittest.TestSuite()
-    suite.addTests(doctest.DocTestSuite(Translator.__module__))
+    suite.addTest(doctest.DocTestSuite(Translator.__module__))
     suite.addTest(unittest.makeSuite(TranslatorTestCase, 'test'))
     suite.addTest(unittest.makeSuite(ExtractTestCase, 'test'))
     return suite
--- a/genshi/filters/tests/transform.py
+++ b/genshi/filters/tests/transform.py
@@ -19,9 +19,12 @@
 
 def suite():
     from genshi.input import HTML
+    from genshi.core import Markup
+    from genshi.builder import tag
     suite = doctest.DocTestSuite(genshi.filters.transform,
                                  optionflags=doctest.NORMALIZE_WHITESPACE,
-                                 extraglobs={'HTML': HTML})
+                                 extraglobs={'HTML': HTML, 'tag': tag,
+                                     'Markup': Markup})
     return suite
 
 if __name__ == '__main__':
--- a/genshi/filters/transform.py
+++ b/genshi/filters/transform.py
@@ -43,13 +43,15 @@
 
 The ``Transformer`` support a large number of useful transformations out of the
 box, but custom transformations can be added easily.
+
+:since: version 0.5
 """
 
 import re
 import sys
 
 from genshi.builder import Element
-from genshi.core import Stream, Attrs, QName, TEXT, START, END, _ensure
+from genshi.core import Stream, Attrs, QName, TEXT, START, END, _ensure, Markup
 from genshi.path import Path
 
 __all__ = ['Transformer', 'StreamBuffer', 'InjectorTransformation', 'ENTER',
@@ -141,14 +143,12 @@
 
     __slots__ = ['transforms']
 
-    def __init__(self, path=None):
+    def __init__(self, path='.'):
         """Construct a new transformation filter.
 
         :param path: an XPath expression (as string) or a `Path` instance
         """
-        self.transforms = []
-        if path:
-            self.transforms.append(SelectTransformation(path))
+        self.transforms = [SelectTransformation(path)]
 
     def __call__(self, stream):
         """Apply the transform filter to the marked stream.
@@ -160,7 +160,8 @@
         transforms = self._mark(stream)
         for link in self.transforms:
             transforms = link(transforms)
-        return Stream(self._unmark(transforms))
+        return Stream(self._unmark(transforms),
+                      serializer=getattr(stream, 'serializer', None))
 
     def apply(self, function):
         """Apply a transformation to the stream.
@@ -548,6 +549,10 @@
         ...             '<b>some bold text</b></body></html>')
         >>> print html | Transformer('body').substitute('(?i)some', 'SOME')
         <html><body>SOME text, some more text and <b>SOME bold text</b></body></html>
+        >>> tags = tag.html(tag.body('Some text, some more text and ',
+        ...      Markup('<b>some bold text</b>')))
+        >>> print tags.generate() | Transformer('body').substitute('(?i)some', 'SOME')
+        <html><body>SOME text, some more text and <b>SOME bold text</b></body></html>
 
         :param pattern: A regular expression object or string.
         :param replace: Replacement pattern.
@@ -556,6 +561,16 @@
         """
         return self.apply(SubstituteTransformation(pattern, replace, count))
 
+    def rename(self, name):
+        """Rename matching elements.
+
+        >>> html = HTML('<html><body>Some text, some more text and '
+        ...             '<b>some bold text</b></body></html>')
+        >>> print html | Transformer('body/b').rename('strong')
+        <html><body>Some text, some more text and <strong>some bold text</strong></body></html>
+        """
+        return self.apply(RenameTransformation(name))
+
     def trace(self, prefix='', fileobj=None):
         """Print events as they pass through the transform.
 
@@ -732,7 +747,10 @@
                     yield None, prefix
                 yield mark, event
                 while True:
-                    mark, event = stream.next()
+                    try:
+                        mark, event = stream.next()
+                    except StopIteration:
+                        yield None, element[-1]
                     if not mark:
                         break
                     yield mark, event
@@ -791,8 +809,8 @@
             if mark:
                 queue.append(event)
             else:
-                for event in flush(queue):
-                    yield event
+                for queue_event in flush(queue):
+                    yield queue_event
                 yield None, event
         for event in flush(queue):
             yield event
@@ -851,7 +869,33 @@
         """
         for mark, (kind, data, pos) in stream:
             if kind is TEXT:
-                data = self.pattern.sub(self.replace, data, self.count)
+                new_data = self.pattern.sub(self.replace, data, self.count)
+                if isinstance(data, Markup):
+                    data = Markup(new_data)
+                else:
+                    data = new_data
+            yield mark, (kind, data, pos)
+
+
+class RenameTransformation(object):
+    """Rename matching elements."""
+    def __init__(self, name):
+        """Create the transform.
+
+        :param name: New element name.
+        """
+        self.name = QName(name)
+
+    def __call__(self, stream):
+        """Apply the transform filter to the marked stream.
+
+        :param stream: The marked event stream to filter
+        """
+        for mark, (kind, data, pos) in stream:
+            if mark is ENTER:
+                data = self.name, data[1]
+            elif mark is EXIT:
+                data = self.name
             yield mark, (kind, data, pos)
 
 
@@ -912,9 +956,15 @@
         :param stream: The marked event stream to filter
         """
         for mark, event in stream:
-            if mark in (ENTER, OUTSIDE):
+            if mark is not None:
                 for subevent in self._inject():
                     yield subevent
+                yield mark, event
+                while True:
+                    mark, event = stream.next()
+                    if not mark:
+                        break
+                    yield mark, event
             yield mark, event
 
 
@@ -930,7 +980,10 @@
             yield mark, event
             if mark:
                 while True:
-                    mark, event = stream.next()
+                    try:
+                        mark, event = stream.next()
+                    except StopIteration:
+                        break
                     if not mark:
                         break
                     yield mark, event
--- a/genshi/output.py
+++ b/genshi/output.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -30,7 +30,7 @@
            'XHTMLSerializer', 'HTMLSerializer', 'TextSerializer']
 __docformat__ = 'restructuredtext en'
 
-def encode(iterator, method='xml', encoding='utf-8'):
+def encode(iterator, method='xml', encoding='utf-8', out=None):
     """Encode serializer output into a string.
     
     :param iterator: the iterator returned from serializing a stream (basically
@@ -39,16 +39,27 @@
                    representable in the specified encoding are treated
     :param encoding: how the output string should be encoded; if set to `None`,
                      this method returns a `unicode` object
-    :return: a string or unicode object (depending on the `encoding` parameter)
+    :param out: a file-like object that the output should be written to
+                instead of being returned as one big string; note that if
+                this is a file or socket (or similar), the `encoding` must
+                not be `None` (that is, the output must be encoded)
+    :return: a `str` or `unicode` object (depending on the `encoding`
+             parameter), or `None` if the `out` parameter is provided
+    
     :since: version 0.4.1
+    :note: Changed in 0.5: added the `out` parameter
     """
-    output = u''.join(list(iterator))
     if encoding is not None:
         errors = 'replace'
         if method != 'text' and not isinstance(method, TextSerializer):
             errors = 'xmlcharrefreplace'
-        return output.encode(encoding, errors)
-    return output
+        _encode = lambda string: string.encode(encoding, errors)
+    else:
+        _encode = lambda string: string
+    if out is None:
+        return _encode(u''.join(list(iterator)))
+    for chunk in iterator:
+        out.write(_encode(chunk))
 
 def get_serializer(method='xml', **kwargs):
     """Return a serializer object for the given method.
@@ -103,6 +114,20 @@
     )
     XHTML = XHTML_STRICT
 
+    SVG_FULL = (
+        'svg', '-//W3C//DTD SVG 1.1//EN',
+        'http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd'
+    )
+    SVG_BASIC = (
+        'svg', '-//W3C//DTD SVG Basic 1.1//EN',
+        'http://www.w3.org/Graphics/SVG/1.1/DTD/svg11-basic.dtd'
+    )
+    SVG_TINY = (
+        'svg', '-//W3C//DTD SVG Tiny 1.1//EN',
+        'http://www.w3.org/Graphics/SVG/1.1/DTD/svg11-tiny.dtd'
+    )
+    SVG = SVG_FULL
+
     def get(cls, name):
         """Return the ``(name, pubid, sysid)`` tuple of the ``DOCTYPE``
         declaration for the specified name.
@@ -115,6 +140,9 @@
          * "xhtml" or "xhtml-strict" for the XHTML 1.0 strict DTD
          * "xhtml-transitional" for the XHTML 1.0 transitional DTD
          * "xhtml-frameset" for the XHTML 1.0 frameset DTD
+         * "svg" or "svg-full" for the SVG 1.1 DTD
+         * "svg-basic" for the SVG Basic 1.1 DTD
+         * "svg-tiny" for the SVG Tiny 1.1 DTD
         
         :param name: the name of the ``DOCTYPE``
         :return: the ``(name, pubid, sysid)`` tuple for the requested
@@ -129,6 +157,9 @@
             'xhtml': cls.XHTML, 'xhtml-strict': cls.XHTML_STRICT,
             'xhtml-transitional': cls.XHTML_TRANSITIONAL,
             'xhtml-frameset': cls.XHTML_FRAMESET,
+            'svg': cls.SVG, 'svg-full': cls.SVG_FULL,
+            'svg-basic': cls.SVG_BASIC,
+            'svg-tiny': cls.SVG_TINY
         }.get(name.lower())
     get = classmethod(get)
 
@@ -156,21 +187,17 @@
                                  stripped from the output
         :note: Changed in 0.4.2: The  `doctype` parameter can now be a string.
         """
-        self.preamble = []
-        if doctype:
-            if isinstance(doctype, basestring):
-                doctype = DocType.get(doctype)
-            self.preamble.append((DOCTYPE, doctype, (None, -1, -1)))
         self.filters = [EmptyTagFilter()]
         if strip_whitespace:
             self.filters.append(WhitespaceFilter(self._PRESERVE_SPACE))
         self.filters.append(NamespaceFlattener(prefixes=namespace_prefixes))
+        if doctype:
+            self.filters.append(DocTypeInserter(doctype))
 
     def __call__(self, stream):
         have_decl = have_doctype = False
         in_cdata = False
 
-        stream = chain(self.preamble, stream)
         for filter_ in self.filters:
             stream = filter_(stream)
         for kind, data, pos in stream:
@@ -217,7 +244,7 @@
                 if sysid:
                     buf.append(' "%s"')
                 buf.append('>\n')
-                yield Markup(u''.join(buf), *filter(None, data))
+                yield Markup(u''.join(buf)) % filter(None, data)
                 have_doctype = True
 
             elif kind is START_CDATA:
@@ -261,6 +288,8 @@
         namespace_prefixes = namespace_prefixes or {}
         namespace_prefixes['http://www.w3.org/1999/xhtml'] = ''
         self.filters.append(NamespaceFlattener(prefixes=namespace_prefixes))
+        if doctype:
+            self.filters.append(DocTypeInserter(doctype))
 
     def __call__(self, stream):
         boolean_attrs = self._BOOLEAN_ATTRS
@@ -268,7 +297,6 @@
         have_doctype = False
         in_cdata = False
 
-        stream = chain(self.preamble, stream)
         for filter_ in self.filters:
             stream = filter_(stream)
         for kind, data, pos in stream:
@@ -281,6 +309,8 @@
                         value = attr
                     elif attr == u'xml:lang' and u'lang' not in attrib:
                         buf += [' lang="', escape(value), '"']
+                    elif attr == u'xml:space':
+                        continue
                     buf += [' ', attr, '="', escape(value), '"']
                 if kind is EMPTY:
                     if tag in empty_elems:
@@ -313,7 +343,7 @@
                 if sysid:
                     buf.append(' "%s"')
                 buf.append('>\n')
-                yield Markup(u''.join(buf), *filter(None, data))
+                yield Markup(u''.join(buf)) % filter(None, data)
                 have_doctype = True
 
             elif kind is START_CDATA:
@@ -359,6 +389,8 @@
         self.filters.append(NamespaceFlattener(prefixes={
             'http://www.w3.org/1999/xhtml': ''
         }))
+        if doctype:
+            self.filters.append(DocTypeInserter(doctype))
 
     def __call__(self, stream):
         boolean_attrs = self._BOOLEAN_ATTRS
@@ -367,7 +399,6 @@
         have_doctype = False
         noescape = False
 
-        stream = chain(self.preamble, stream)
         for filter_ in self.filters:
             stream = filter_(stream)
         for kind, data, pos in stream:
@@ -415,7 +446,7 @@
                 if sysid:
                     buf.append(' "%s"')
                 buf.append('>\n')
-                yield Markup(u''.join(buf), *filter(None, data))
+                yield Markup(u''.join(buf)) % filter(None, data)
                 have_doctype = True
 
             elif kind is PI:
@@ -436,20 +467,29 @@
     <Hello!>
 
     If text events contain literal markup (instances of the `Markup` class),
-    tags or entities are stripped from the output:
+    that markup is by default passed through unchanged:
     
-    >>> elem = tag.div(Markup('<a href="foo">Hello!</a><br/>'))
-    >>> print elem
-    <div><a href="foo">Hello!</a><br/></div>
-    >>> print ''.join(TextSerializer()(elem.generate()))
-    Hello!
+    >>> elem = tag.div(Markup('<a href="foo">Hello &amp; Bye!</a><br/>'))
+    >>> print elem.generate().render(TextSerializer)
+    <a href="foo">Hello &amp; Bye!</a><br/>
+    
+    You can use the `strip_markup` to change this behavior, so that tags and
+    entities are stripped from the output (or in the case of entities,
+    replaced with the equivalent character):
+
+    >>> print elem.generate().render(TextSerializer, strip_markup=True)
+    Hello & Bye!
     """
 
+    def __init__(self, strip_markup=False):
+        self.strip_markup = strip_markup
+
     def __call__(self, stream):
+        strip_markup = self.strip_markup
         for event in stream:
             if event[0] is TEXT:
                 data = event[1]
-                if type(data) is Markup:
+                if strip_markup and type(data) is Markup:
                     data = data.striptags().stripentities()
                 yield unicode(data)
 
@@ -667,3 +707,33 @@
 
                 if kind:
                     yield kind, data, pos
+
+
+class DocTypeInserter(object):
+    """A filter that inserts the DOCTYPE declaration in the correct location,
+    after the XML declaration.
+    """
+    def __init__(self, doctype):
+        """Initialize the filter.
+
+        :param doctype: DOCTYPE as a string or DocType object.
+        """
+        if isinstance(doctype, basestring):
+            doctype = DocType.get(doctype)
+        self.doctype_event = (DOCTYPE, doctype, (None, -1, -1))
+
+    def __call__(self, stream):
+        doctype_inserted = False
+        for kind, data, pos in stream:
+            if not doctype_inserted:
+                doctype_inserted = True
+                if kind is XML_DECL:
+                    yield (kind, data, pos)
+                    yield self.doctype_event
+                    continue
+                yield self.doctype_event
+
+            yield (kind, data, pos)
+
+        if not doctype_inserted:
+            yield self.doctype_event
--- a/genshi/path.py
+++ b/genshi/path.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -39,10 +39,12 @@
 """
 
 from math import ceil, floor
+import operator
 import re
 
 from genshi.core import Stream, Attrs, Namespace, QName
-from genshi.core import START, END, TEXT, COMMENT, PI
+from genshi.core import START, END, TEXT, START_NS, END_NS, COMMENT, PI, \
+                        START_CDATA, END_CDATA
 
 __all__ = ['Path', 'PathSyntaxError']
 __docformat__ = 'restructuredtext en'
@@ -146,7 +148,8 @@
                                  updateonly=True)
                 elif result:
                     yield result
-        return Stream(_generate())
+        return Stream(_generate(),
+                      serializer=getattr(stream, 'serializer', None))
 
     def test(self, ignore_context=False):
         """Returns a function that can be used to track whether the path matches
@@ -193,6 +196,9 @@
                     continue
                 elif kind is START:
                     cursors.append(cursors and cursors[-1] or 0)
+                elif kind is START_NS or kind is END_NS \
+                        or kind is START_CDATA or kind is END_CDATA:
+                    continue
 
                 if updateonly or retval or not cursors:
                     continue
@@ -270,6 +276,8 @@
                     if not matched or last_step or not (
                             axis is SELF or axis is DESCENDANT_OR_SELF):
                         break
+                    if ctxtnode and axis is DESCENDANT_OR_SELF:
+                        ctxtnode = False
 
                 if (retval or not matched) and kind is START and \
                         not (axis is DESCENDANT or axis is DESCENDANT_OR_SELF):
@@ -771,7 +779,7 @@
         string2 = as_string(self.string2(kind, data, pos, namespaces, variables))
         return re.search(string2, string1, self.flags)
     def _map_flags(self, flags):
-        return reduce(lambda a, b: a | b,
+        return reduce(operator.or_,
                       [self.flag_map[flag] for flag in flags], re.U)
     def __repr__(self):
         return 'contains(%r, %r)' % (self.string1, self.string2)
--- a/genshi/template/__init__.py
+++ b/genshi/template/__init__.py
@@ -18,6 +18,6 @@
                                  BadDirectiveError
 from genshi.template.loader import TemplateLoader, TemplateNotFound
 from genshi.template.markup import MarkupTemplate
-from genshi.template.text import TextTemplate
+from genshi.template.text import TextTemplate, OldTextTemplate, NewTextTemplate
 
 __docformat__ = 'restructuredtext en'
--- a/genshi/template/base.py
+++ b/genshi/template/base.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -15,6 +15,7 @@
 
 import os
 from StringIO import StringIO
+import sys
 
 from genshi.core import Attrs, Stream, StreamEventKind, START, TEXT, _ensure
 from genshi.input import ParseError
@@ -27,7 +28,7 @@
 class TemplateError(Exception):
     """Base exception class for errors related to template processing."""
 
-    def __init__(self, message, filename='<string>', lineno=-1, offset=-1):
+    def __init__(self, message, filename=None, lineno=-1, offset=-1):
         """Create the exception.
         
         :param message: the error message
@@ -36,6 +37,8 @@
                        occurred
         :param offset: the column number at which the error occurred
         """
+        if filename is None:
+            filename = '<string>'
         self.msg = message #: the error message string
         if filename != '<string>' or lineno >= 0:
             message = '%s (%s, line %d)' % (self.msg, filename, lineno)
@@ -50,7 +53,7 @@
     error, or the template is not well-formed.
     """
 
-    def __init__(self, message, filename='<string>', lineno=-1, offset=-1):
+    def __init__(self, message, filename=None, lineno=-1, offset=-1):
         """Create the exception
         
         :param message: the error message
@@ -72,7 +75,7 @@
     with a local name that doesn't match any registered directive.
     """
 
-    def __init__(self, name, filename='<string>', lineno=-1):
+    def __init__(self, name, filename=None, lineno=-1):
         """Create the exception
         
         :param name: the name of the directive
@@ -169,19 +172,54 @@
         self.pop()
         self.push(data)
 
-
-def _apply_directives(stream, ctxt, directives):
+def _apply_directives(stream, directives, ctxt, **vars):
     """Apply the given directives to the stream.
     
     :param stream: the stream the directives should be applied to
+    :param directives: the list of directives to apply
     :param ctxt: the `Context`
-    :param directives: the list of directives to apply
+    :param vars: additional variables that should be available when Python
+                 code is executed
     :return: the stream with the given directives applied
     """
     if directives:
-        stream = directives[0](iter(stream), ctxt, directives[1:])
+        stream = directives[0](iter(stream), directives[1:], ctxt, **vars)
     return stream
 
+def _eval_expr(expr, ctxt, **vars):
+    """Evaluate the given `Expression` object.
+    
+    :param expr: the expression to evaluate
+    :param ctxt: the `Context`
+    :param vars: additional variables that should be available to the
+                 expression
+    :return: the result of the evaluation
+    """
+    if vars:
+        ctxt.push(vars)
+    retval = expr.evaluate(ctxt.data)
+    if vars:
+        ctxt.pop()
+    return retval
+
+def _exec_suite(suite, ctxt, **vars):
+    """Execute the given `Suite` object.
+    
+    :param suite: the code suite to execute
+    :param ctxt: the `Context`
+    :param vars: additional variables that should be available to the
+                 code
+    """
+    frame = {}
+    if vars:
+        ctxt.push(vars)
+        ctxt.push(frame)
+    suite.execute(ctxt.data)
+    if vars:
+        ctxt.pop()
+        ctxt.pop()
+        ctxt.data.update(frame)
+
 
 class TemplateMeta(type):
     """Meta class for templates."""
@@ -202,6 +240,9 @@
     """
     __metaclass__ = TemplateMeta
 
+    EXEC = StreamEventKind('EXEC')
+    """Stream event kind representing a Python code suite to execute."""
+
     EXPR = StreamEventKind('EXPR')
     """Stream event kind representing a Python expression."""
 
@@ -213,38 +254,35 @@
     directives should be applied.
     """
 
-    def __init__(self, source, basedir=None, filename=None, loader=None,
-                 encoding=None, lookup='lenient', allow_exec=True):
+    serializer = None
+    _number_conv = unicode # function used to convert numbers to event data
+
+    def __init__(self, source, filepath=None, filename=None, loader=None,
+                 encoding=None, lookup='strict', allow_exec=True):
         """Initialize a template from either a string, a file-like object, or
         an already parsed markup stream.
         
         :param source: a string, file-like object, or markup stream to read the
                        template from
-        :param basedir: the base directory containing the template file; when
-                        loaded from a `TemplateLoader`, this will be the
-                        directory on the template search path in which the
-                        template was found
-        :param filename: the name of the template file, relative to the given
-                         base directory
+        :param filepath: the absolute path to the template file
+        :param filename: the path to the template file relative to the search
+                         path
         :param loader: the `TemplateLoader` to use for loading included
                        templates
         :param encoding: the encoding of the `source`
-        :param lookup: the variable lookup mechanism; either "lenient" (the
-                       default), "strict", or a custom lookup class
+        :param lookup: the variable lookup mechanism; either "strict" (the
+                       default), "lenient", or a custom lookup class
         :param allow_exec: whether Python code blocks in templates should be
                            allowed
         
         :note: Changed in 0.5: Added the `allow_exec` argument
         """
-        self.basedir = basedir
+        self.filepath = filepath or filename
         self.filename = filename
-        if basedir and filename:
-            self.filepath = os.path.join(basedir, filename)
-        else:
-            self.filepath = filename
         self.loader = loader
         self.lookup = lookup
         self.allow_exec = allow_exec
+        self._init_filters()
 
         if isinstance(source, basestring):
             source = StringIO(source)
@@ -254,13 +292,24 @@
             self.stream = list(self._prepare(self._parse(source, encoding)))
         except ParseError, e:
             raise TemplateSyntaxError(e.msg, self.filepath, e.lineno, e.offset)
-        self.filters = [self._flatten, self._eval]
-        if loader:
-            self.filters.append(self._include)
+
+    def __getstate__(self):
+        state = self.__dict__.copy()
+        state['filters'] = []
+        return state
+
+    def __setstate__(self, state):
+        self.__dict__ = state
+        self._init_filters()
 
     def __repr__(self):
         return '<%s "%s">' % (self.__class__.__name__, self.filename)
 
+    def _init_filters(self):
+        self.filters = [self._flatten, self._eval, self._exec]
+        if self.loader:
+            self.filters.append(self._include)
+
     def _parse(self, source, encoding):
         """Parse the template.
         
@@ -299,7 +348,7 @@
                         yield event
             else:
                 if kind is INCLUDE:
-                    href, fallback = data
+                    href, cls, fallback = data
                     if isinstance(href, basestring) and \
                             not getattr(self.loader, 'auto_reload', True):
                         # If the path to the included template is static, and
@@ -307,7 +356,7 @@
                         # the template is inlined into the stream
                         try:
                             tmpl = self.loader.load(href, relative_to=pos[0],
-                                                    cls=self.__class__)
+                                                    cls=cls or self.__class__)
                             for event in tmpl.stream:
                                 yield event
                         except TemplateNotFound:
@@ -316,9 +365,9 @@
                             for event in self._prepare(fallback):
                                 yield event
                         continue
-                    else:
+                    elif fallback:
                         # Otherwise the include is performed at run time
-                        data = href, list(self._prepare(fallback))
+                        data = href, cls, list(self._prepare(fallback))
 
                 yield kind, data, pos
 
@@ -335,25 +384,29 @@
         :return: a markup event stream representing the result of applying
                  the template to the context data.
         """
+        vars = {}
         if args:
             assert len(args) == 1
             ctxt = args[0]
             if ctxt is None:
                 ctxt = Context(**kwargs)
+            else:
+                vars = kwargs
             assert isinstance(ctxt, Context)
         else:
             ctxt = Context(**kwargs)
 
         stream = self.stream
         for filter_ in self.filters:
-            stream = filter_(iter(stream), ctxt)
-        return Stream(stream)
+            stream = filter_(iter(stream), ctxt, **vars)
+        return Stream(stream, self.serializer)
 
-    def _eval(self, stream, ctxt):
+    def _eval(self, stream, ctxt, **vars):
         """Internal stream filter that evaluates any expressions in `START` and
         `TEXT` events.
         """
         filters = (self._flatten, self._eval)
+        number_conv = self._number_conv
 
         for kind, data, pos in stream:
 
@@ -368,7 +421,8 @@
                     else:
                         values = []
                         for subkind, subdata, subpos in self._eval(substream,
-                                                                   ctxt):
+                                                                   ctxt,
+                                                                   **vars):
                             if subkind is TEXT:
                                 values.append(subdata)
                         value = [x for x in values if x is not None]
@@ -378,17 +432,19 @@
                 yield kind, (tag, Attrs(new_attrs)), pos
 
             elif kind is EXPR:
-                result = data.evaluate(ctxt.data)
+                result = _eval_expr(data, ctxt, **vars)
                 if result is not None:
-                    # First check for a string, otherwise the iterable test below
-                    # succeeds, and the string will be chopped up into individual
-                    # characters
+                    # First check for a string, otherwise the iterable test
+                    # below succeeds, and the string will be chopped up into
+                    # individual characters
                     if isinstance(result, basestring):
                         yield TEXT, result, pos
+                    elif isinstance(result, (int, float, long)):
+                        yield TEXT, number_conv(result), pos
                     elif hasattr(result, '__iter__'):
                         substream = _ensure(result)
                         for filter_ in filters:
-                            substream = filter_(substream, ctxt)
+                            substream = filter_(substream, ctxt, **vars)
                         for event in substream:
                             yield event
                     else:
@@ -397,20 +453,29 @@
             else:
                 yield kind, data, pos
 
-    def _flatten(self, stream, ctxt):
+    def _exec(self, stream, ctxt, **vars):
+        """Internal stream filter that executes Python code blocks."""
+        for event in stream:
+            if event[0] is EXEC:
+                _exec_suite(event[1], ctxt, **vars)
+            else:
+                yield event
+
+    def _flatten(self, stream, ctxt, **vars):
         """Internal stream filter that expands `SUB` events in the stream."""
         for event in stream:
             if event[0] is SUB:
                 # This event is a list of directives and a list of nested
                 # events to which those directives should be applied
                 directives, substream = event[1]
-                substream = _apply_directives(substream, ctxt, directives)
-                for event in self._flatten(substream, ctxt):
+                substream = _apply_directives(substream, directives, ctxt,
+                                              **vars)
+                for event in self._flatten(substream, ctxt, **vars):
                     yield event
             else:
                 yield event
 
-    def _include(self, stream, ctxt):
+    def _include(self, stream, ctxt, **vars):
         """Internal stream filter that performs inclusion of external
         template files.
         """
@@ -418,29 +483,31 @@
 
         for event in stream:
             if event[0] is INCLUDE:
-                href, fallback = event[1]
+                href, cls, fallback = event[1]
                 if not isinstance(href, basestring):
                     parts = []
-                    for subkind, subdata, subpos in self._eval(href, ctxt):
+                    for subkind, subdata, subpos in self._eval(href, ctxt,
+                                                               **vars):
                         if subkind is TEXT:
                             parts.append(subdata)
                     href = u''.join([x for x in parts if x is not None])
                 try:
                     tmpl = self.loader.load(href, relative_to=event[2][0],
-                                            cls=self.__class__)
-                    for event in tmpl.generate(ctxt):
+                                            cls=cls or self.__class__)
+                    for event in tmpl.generate(ctxt, **vars):
                         yield event
                 except TemplateNotFound:
                     if fallback is None:
                         raise
                     for filter_ in self.filters:
-                        fallback = filter_(iter(fallback), ctxt)
+                        fallback = filter_(iter(fallback), ctxt, **vars)
                     for event in fallback:
                         yield event
             else:
                 yield event
 
 
+EXEC = Template.EXEC
 EXPR = Template.EXPR
 INCLUDE = Template.INCLUDE
 SUB = Template.SUB
--- a/genshi/template/directives.py
+++ b/genshi/template/directives.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -14,12 +14,17 @@
 """Implementation of the various template directives."""
 
 import compiler
+try:
+    frozenset
+except NameError:
+    from sets import ImmutableSet as frozenset
 
 from genshi.core import QName, Stream
 from genshi.path import Path
 from genshi.template.base import TemplateRuntimeError, TemplateSyntaxError, \
-                                 EXPR, _apply_directives
-from genshi.template.eval import Expression, _parse
+                                 EXPR, _apply_directives, _eval_expr, \
+                                 _exec_suite
+from genshi.template.eval import Expression, ExpressionASTTransformer, _parse
 
 __all__ = ['AttrsDirective', 'ChooseDirective', 'ContentDirective',
            'DefDirective', 'ForDirective', 'IfDirective', 'MatchDirective',
@@ -83,13 +88,15 @@
         return cls(value, template, namespaces, *pos[1:]), stream
     attach = classmethod(attach)
 
-    def __call__(self, stream, ctxt, directives):
+    def __call__(self, stream, directives, ctxt, **vars):
         """Apply the directive to the given stream.
         
         :param stream: the event stream
-        :param ctxt: the context data
         :param directives: a list of the remaining directives that should
                            process the stream
+        :param ctxt: the context data
+        :param vars: additional variables that should be made available when
+                     Python code is executed
         """
         raise NotImplementedError
 
@@ -162,10 +169,10 @@
     """
     __slots__ = []
 
-    def __call__(self, stream, ctxt, directives):
+    def __call__(self, stream, directives, ctxt, **vars):
         def _generate():
             kind, (tag, attrib), pos  = stream.next()
-            attrs = self.expr.evaluate(ctxt.data)
+            attrs = _eval_expr(self.expr, ctxt, **vars)
             if attrs:
                 if isinstance(attrs, Stream):
                     try:
@@ -181,7 +188,7 @@
             for event in stream:
                 yield event
 
-        return _apply_directives(_generate(), ctxt, directives)
+        return _apply_directives(_generate(), directives, ctxt, **vars)
 
 
 class ContentDirective(Directive):
@@ -202,6 +209,10 @@
     __slots__ = []
 
     def attach(cls, template, stream, value, namespaces, pos):
+        if type(value) is dict:
+            raise TemplateSyntaxError('The content directive can not be used '
+                                      'as an element', template.filepath,
+                                      *pos[1:])
         expr = cls._parse_expr(value, template, *pos[1:])
         return None, [stream[0], (EXPR, expr, pos),  stream[-1]]
     attach = classmethod(attach)
@@ -282,7 +293,7 @@
                                                namespaces, pos)
     attach = classmethod(attach)
 
-    def __call__(self, stream, ctxt, directives):
+    def __call__(self, stream, directives, ctxt, **vars):
         stream = list(stream)
 
         def function(*args, **kwargs):
@@ -295,14 +306,14 @@
                     if name in kwargs:
                         val = kwargs.pop(name)
                     else:
-                        val = self.defaults.get(name).evaluate(ctxt.data)
+                        val = _eval_expr(self.defaults.get(name), ctxt, **vars)
                     scope[name] = val
             if not self.star_args is None:
                 scope[self.star_args] = args
             if not self.dstar_args is None:
                 scope[self.dstar_args] = kwargs
             ctxt.push(scope)
-            for event in _apply_directives(stream, ctxt, directives):
+            for event in _apply_directives(stream, directives, ctxt, **vars):
                 yield event
             ctxt.pop()
         try:
@@ -343,10 +354,10 @@
                                       template.filepath, lineno, offset)
         assign, value = value.split(' in ', 1)
         ast = _parse(assign, 'exec')
+        value = 'iter(%s)' % value.strip()
         self.assign = _assignment(ast.node.nodes[0].expr)
         self.filename = template.filepath
-        Directive.__init__(self, value.strip(), template, namespaces, lineno,
-                           offset)
+        Directive.__init__(self, value, template, namespaces, lineno, offset)
 
     def attach(cls, template, stream, value, namespaces, pos):
         if type(value) is dict:
@@ -355,24 +366,20 @@
                                                namespaces, pos)
     attach = classmethod(attach)
 
-    def __call__(self, stream, ctxt, directives):
-        iterable = self.expr.evaluate(ctxt.data)
+    def __call__(self, stream, directives, ctxt, **vars):
+        iterable = _eval_expr(self.expr, ctxt, **vars)
         if iterable is None:
             return
 
         assign = self.assign
         scope = {}
         stream = list(stream)
-        try:
-            iterator = iter(iterable)
-            for item in iterator:
-                assign(scope, item)
-                ctxt.push(scope)
-                for event in _apply_directives(stream, ctxt, directives):
-                    yield event
-                ctxt.pop()
-        except TypeError, e:
-            raise TemplateRuntimeError(str(e), self.filename, *stream[0][2][1:])
+        for item in iterable:
+            assign(scope, item)
+            ctxt.push(scope)
+            for event in _apply_directives(stream, directives, ctxt, **vars):
+                yield event
+            ctxt.pop()
 
     def __repr__(self):
         return '<%s>' % self.__class__.__name__
@@ -400,9 +407,10 @@
                                               namespaces, pos)
     attach = classmethod(attach)
 
-    def __call__(self, stream, ctxt, directives):
-        if self.expr.evaluate(ctxt.data):
-            return _apply_directives(stream, ctxt, directives)
+    def __call__(self, stream, directives, ctxt, **vars):
+        value = _eval_expr(self.expr, ctxt, **vars)
+        if value:
+            return _apply_directives(stream, directives, ctxt, **vars)
         return []
 
 
@@ -423,24 +431,33 @@
       </span>
     </div>
     """
-    __slots__ = ['path', 'namespaces']
+    __slots__ = ['path', 'namespaces', 'hints']
 
-    def __init__(self, value, template, namespaces=None, lineno=-1, offset=-1):
+    def __init__(self, value, template, hints=None, namespaces=None,
+                 lineno=-1, offset=-1):
         Directive.__init__(self, None, template, namespaces, lineno, offset)
         self.path = Path(value, template.filepath, lineno)
         self.namespaces = namespaces or {}
+        self.hints = hints or ()
 
     def attach(cls, template, stream, value, namespaces, pos):
+        hints = []
         if type(value) is dict:
+            if value.get('buffer', '').lower() == 'false':
+                hints.append('not_buffered')
+            if value.get('once', '').lower() == 'true':
+                hints.append('match_once')
+            if value.get('recursive', '').lower() == 'false':
+                hints.append('not_recursive')
             value = value.get('path')
-        return super(MatchDirective, cls).attach(template, stream, value,
-                                                 namespaces, pos)
+        return cls(value, template, frozenset(hints), namespaces, *pos[1:]), \
+               stream
     attach = classmethod(attach)
 
-    def __call__(self, stream, ctxt, directives):
+    def __call__(self, stream, directives, ctxt, **vars):
         ctxt._match_templates.append((self.path.test(ignore_context=True),
-                                      self.path, list(stream), self.namespaces,
-                                      directives))
+                                      self.path, list(stream), self.hints,
+                                      self.namespaces, directives))
         return []
 
     def __repr__(self):
@@ -476,6 +493,8 @@
     __slots__ = []
 
     def attach(cls, template, stream, value, namespaces, pos):
+        if type(value) is dict:
+            value = value.get('value')
         if not value:
             raise TemplateSyntaxError('missing value for "replace" directive',
                                       template.filepath, *pos[1:])
@@ -517,9 +536,9 @@
     """
     __slots__ = []
 
-    def __call__(self, stream, ctxt, directives):
+    def __call__(self, stream, directives, ctxt, **vars):
         def _generate():
-            if self.expr.evaluate(ctxt.data):
+            if _eval_expr(self.expr, ctxt, **vars):
                 stream.next() # skip start tag
                 previous = stream.next()
                 for event in stream:
@@ -528,7 +547,7 @@
             else:
                 for event in stream:
                     yield event
-        return _apply_directives(_generate(), ctxt, directives)
+        return _apply_directives(_generate(), directives, ctxt, **vars)
 
     def attach(cls, template, stream, value, namespaces, pos):
         if not value:
@@ -586,12 +605,12 @@
                                                   namespaces, pos)
     attach = classmethod(attach)
 
-    def __call__(self, stream, ctxt, directives):
+    def __call__(self, stream, directives, ctxt, **vars):
         info = [False, bool(self.expr), None]
         if self.expr:
             info[2] = self.expr.evaluate(ctxt.data)
         ctxt._choice_stack.append(info)
-        for event in _apply_directives(stream, ctxt, directives):
+        for event in _apply_directives(stream, directives, ctxt, **vars):
             yield event
         ctxt._choice_stack.pop()
 
@@ -615,7 +634,7 @@
                                                 namespaces, pos)
     attach = classmethod(attach)
 
-    def __call__(self, stream, ctxt, directives):
+    def __call__(self, stream, directives, ctxt, **vars):
         info = ctxt._choice_stack and ctxt._choice_stack[-1]
         if not info:
             raise TemplateRuntimeError('"when" directives can only be used '
@@ -630,16 +649,16 @@
         if info[1]:
             value = info[2]
             if self.expr:
-                matched = value == self.expr.evaluate(ctxt.data)
+                matched = value == _eval_expr(self.expr, ctxt, **vars)
             else:
                 matched = bool(value)
         else:
-            matched = bool(self.expr.evaluate(ctxt.data))
+            matched = bool(_eval_expr(self.expr, ctxt, **vars))
         info[0] = matched
         if not matched:
             return []
 
-        return _apply_directives(stream, ctxt, directives)
+        return _apply_directives(stream, directives, ctxt, **vars)
 
 
 class OtherwiseDirective(Directive):
@@ -654,7 +673,7 @@
         Directive.__init__(self, None, template, namespaces, lineno, offset)
         self.filename = template.filepath
 
-    def __call__(self, stream, ctxt, directives):
+    def __call__(self, stream, directives, ctxt, **vars):
         info = ctxt._choice_stack and ctxt._choice_stack[-1]
         if not info:
             raise TemplateRuntimeError('an "otherwise" directive can only be '
@@ -664,7 +683,7 @@
             return []
         info[0] = True
 
-        return _apply_directives(stream, ctxt, directives)
+        return _apply_directives(stream, directives, ctxt, **vars)
 
 
 class WithDirective(Directive):
@@ -684,20 +703,20 @@
 
     def __init__(self, value, template, namespaces=None, lineno=-1, offset=-1):
         Directive.__init__(self, None, template, namespaces, lineno, offset)
-        self.vars = []
-        value = value.strip()
+        self.vars = [] 
+        value = value.strip() 
         try:
-            ast = _parse(value, 'exec').node
-            for node in ast.nodes:
-                if isinstance(node, compiler.ast.Discard):
-                    continue
-                elif not isinstance(node, compiler.ast.Assign):
-                    raise TemplateSyntaxError('only assignment allowed in '
-                                              'value of the "with" directive',
-                                              template.filepath, lineno, offset)
-                self.vars.append(([_assignment(n) for n in node.nodes],
-                                  Expression(node.expr, template.filepath,
-                                             lineno, lookup=template.lookup)))
+            ast = _parse(value, 'exec').node 
+            for node in ast.nodes: 
+                if isinstance(node, compiler.ast.Discard): 
+                    continue 
+                elif not isinstance(node, compiler.ast.Assign): 
+                    raise TemplateSyntaxError('only assignment allowed in ' 
+                                              'value of the "with" directive', 
+                                              template.filepath, lineno, offset) 
+                self.vars.append(([_assignment(n) for n in node.nodes], 
+                                  Expression(node.expr, template.filepath, 
+                                             lineno, lookup=template.lookup))) 
         except SyntaxError, err:
             err.msg += ' in expression "%s" of "%s" directive' % (value,
                                                                   self.tagname)
@@ -711,15 +730,16 @@
                                                 namespaces, pos)
     attach = classmethod(attach)
 
-    def __call__(self, stream, ctxt, directives):
+    def __call__(self, stream, directives, ctxt, **vars):
         frame = {}
         ctxt.push(frame)
         for targets, expr in self.vars:
             value = expr.evaluate(ctxt.data)
             for assign in targets:
                 assign(frame, value)
-            ctxt.replace(frame)
-        for event in _apply_directives(stream, ctxt, directives):
+            ctxt.pop()
+            ctxt.push(frame)
+        for event in _apply_directives(stream, directives, ctxt, **vars):
             yield event
         ctxt.pop()
 
--- a/genshi/template/eval.py
+++ b/genshi/template/eval.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -23,6 +23,7 @@
     from sets import ImmutableSet as frozenset
     from sets import Set as set
 import sys
+from textwrap import dedent
 
 from genshi.core import Markup
 from genshi.template.base import TemplateRuntimeError
@@ -37,7 +38,8 @@
     """Abstract base class for the `Expression` and `Suite` classes."""
     __slots__ = ['source', 'code', 'ast', '_globals']
 
-    def __init__(self, source, filename=None, lineno=-1, lookup='lenient'):
+    def __init__(self, source, filename=None, lineno=-1, lookup='strict',
+                 xform=None):
         """Create the code object, either from a string, or from an AST node.
         
         :param source: either a string containing the source code, or an AST
@@ -46,14 +48,18 @@
                          the code
         :param lineno: the number of the line on which the code was found
         :param lookup: the lookup class that defines how variables are looked
-                       up in the context. Can be either `LenientLookup` (the
-                       default), `StrictLookup`, or a custom lookup class
+                       up in the context; can be either "strict" (the default),
+                       "lenient", or a custom lookup class
+        :param xform: the AST transformer that should be applied to the code;
+                      if `None`, the appropriate transformation is chosen
+                      depending on the mode
         """
         if isinstance(source, basestring):
             self.source = source
             node = _parse(source, mode=self.mode)
         else:
-            assert isinstance(source, ast.Node)
+            assert isinstance(source, ast.Node), \
+                'Expected string or AST node, but got %r' % source
             self.source = '?'
             if self.mode == 'eval':
                 node = ast.Expression(source)
@@ -62,12 +68,27 @@
 
         self.ast = node
         self.code = _compile(node, self.source, mode=self.mode,
-                             filename=filename, lineno=lineno)
+                             filename=filename, lineno=lineno, xform=xform)
         if lookup is None:
             lookup = LenientLookup
         elif isinstance(lookup, basestring):
             lookup = {'lenient': LenientLookup, 'strict': StrictLookup}[lookup]
-        self._globals = lookup.globals()
+        self._globals = lookup.globals
+
+    def __getstate__(self):
+        state = {'source': self.source, 'ast': self.ast,
+                 'lookup': self._globals.im_self}
+        c = self.code
+        state['code'] = (c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code,
+                         c.co_consts, c.co_names, c.co_varnames, c.co_filename,
+                         c.co_name, c.co_firstlineno, c.co_lnotab, (), ())
+        return state
+
+    def __setstate__(self, state):
+        self.source = state['source']
+        self.ast = state['ast']
+        self.code = new.code(0, *state['code'])
+        self._globals = state['lookup'].globals
 
     def __eq__(self, other):
         return (type(other) == type(self)) and (self.code == other.code)
@@ -133,8 +154,7 @@
         :return: the result of the evaluation
         """
         __traceback_hide__ = 'before_and_this'
-        _globals = self._globals
-        _globals['data'] = data
+        _globals = self._globals(data)
         return eval(self.code, _globals, data)
 
 
@@ -155,8 +175,7 @@
         :param data: a mapping containing the data to execute in
         """
         __traceback_hide__ = 'before_and_this'
-        _globals = self._globals
-        _globals['data'] = data
+        _globals = self._globals(data)
         exec self.code in _globals, data
 
 
@@ -242,14 +261,16 @@
 class LookupBase(object):
     """Abstract base class for variable lookup implementations."""
 
-    def globals(cls):
+    def globals(cls, data):
         """Construct the globals dictionary to use as the execution context for
         the expression or suite.
         """
         return {
+            '__data__': data,
             '_lookup_name': cls.lookup_name,
             '_lookup_attr': cls.lookup_attr,
-            '_lookup_item': cls.lookup_item
+            '_lookup_item': cls.lookup_item,
+            'UndefinedError': UndefinedError,
         }
     globals = classmethod(globals)
 
@@ -259,18 +280,23 @@
         if val is UNDEFINED:
             val = BUILTINS.get(name, val)
             if val is UNDEFINED:
-                return cls.undefined(name)
+                val = cls.undefined(name)
         return val
     lookup_name = classmethod(lookup_name)
 
     def lookup_attr(cls, obj, key):
         __traceback_hide__ = True
-        if hasattr(obj, key):
-            return getattr(obj, key)
         try:
-            return obj[key]
-        except (KeyError, TypeError):
-            return cls.undefined(key, owner=obj)
+            val = getattr(obj, key)
+        except AttributeError:
+            if hasattr(obj.__class__, key):
+                raise
+            else:
+                try:
+                    val = obj[key]
+                except (KeyError, TypeError):
+                    val = cls.undefined(key, owner=obj)
+        return val
     lookup_attr = classmethod(lookup_attr)
 
     def lookup_item(cls, obj, key):
@@ -283,7 +309,7 @@
             if isinstance(key, basestring):
                 val = getattr(obj, key, UNDEFINED)
                 if val is UNDEFINED:
-                    return cls.undefined(key, owner=obj)
+                    val = cls.undefined(key, owner=obj)
                 return val
             raise
     lookup_item = classmethod(lookup_item)
@@ -358,12 +384,24 @@
 
 
 def _parse(source, mode='eval'):
+    source = source.strip()
+    if mode == 'exec':
+        lines = [line.expandtabs() for line in source.splitlines()]
+        if lines:
+            first = lines[0]
+            rest = dedent('\n'.join(lines[1:])).rstrip()
+            if first.rstrip().endswith(':') and not rest[0].isspace():
+                rest = '\n'.join(['    %s' % line for line in rest.splitlines()])
+            source = '\n'.join([first, rest])
     if isinstance(source, unicode):
         source = '\xef\xbb\xbf' + source.encode('utf-8')
     return parse(source, mode)
 
-def _compile(node, source=None, mode='eval', filename=None, lineno=-1):
-    xform = {'eval': ExpressionASTTransformer}.get(mode, TemplateASTTransformer)
+def _compile(node, source=None, mode='eval', filename=None, lineno=-1,
+             xform=None):
+    if xform is None:
+        xform = {'eval': ExpressionASTTransformer}.get(mode,
+                                                       TemplateASTTransformer)
     tree = xform().visit(node)
     if isinstance(filename, unicode):
         # unicode file names not allowed for code objects
@@ -376,10 +414,17 @@
 
     if mode == 'eval':
         gen = ExpressionCodeGenerator(tree)
-        name = '<Expression %s>' % (repr(source or '?'))
+        name = '<Expression %r>' % (source or '?')
     else:
         gen = ModuleCodeGenerator(tree)
-        name = '<Suite>'
+        lines = source.splitlines()
+        if not lines:
+            extract = ''
+        else:
+            extract = lines[0]
+        if len(lines) > 1:
+            extract += ' ...'
+        name = '<Suite %r>' % (extract)
     gen.optimized = True
     code = gen.getCode()
 
@@ -416,7 +461,8 @@
         node = node.__class__(*args)
         if lineno is not None:
             node.lineno = lineno
-        if isinstance(node, (ast.Class, ast.Function, ast.GenExpr, ast.Lambda)):
+        if isinstance(node, (ast.Class, ast.Function, ast.Lambda)) or \
+                sys.version_info > (2, 4) and isinstance(node, ast.GenExpr):
             node.filename = '<string>' # workaround for bug in pycodegen
         return node
 
@@ -443,7 +489,7 @@
 
     def visitClass(self, node):
         return self._clone(node, node.name, [self.visit(x) for x in node.bases],
-            node.doc, node.code
+            node.doc, self.visit(node.code)
         )
 
     def visitFunction(self, node):
@@ -636,7 +682,7 @@
     """
 
     def __init__(self):
-        self.locals = [CONSTANTS, set()]
+        self.locals = [CONSTANTS]
 
     def visitConst(self, node):
         if isinstance(node.value, str):
@@ -647,18 +693,19 @@
         return node
 
     def visitAssName(self, node):
-        if self.locals:
+        if len(self.locals) > 1:
             self.locals[-1].add(node.name)
         return node
 
     def visitAugAssign(self, node):
-        if isinstance(node.node, ast.Name):
+        if isinstance(node.node, ast.Name) \
+                and node.node.name not in flatten(self.locals):
             name = node.node.name
-            node.node = ast.Subscript(ast.Name('data'), 'OP_APPLY',
+            node.node = ast.Subscript(ast.Name('__data__'), 'OP_APPLY',
                                       [ast.Const(name)])
             node.expr = self.visit(node.expr)
             return ast.If([
-                (ast.Compare(ast.Const(name), [('in', ast.Name('data'))]),
+                (ast.Compare(ast.Const(name), [('in', ast.Name('__data__'))]),
                  ast.Stmt([node]))],
                 ast.Stmt([ast.Raise(ast.CallFunc(ast.Name('UndefinedError'),
                                                  [ast.Const(name)]),
@@ -667,6 +714,8 @@
             return ASTTransformer.visitAugAssign(self, node)
 
     def visitClass(self, node):
+        if len(self.locals) > 1:
+            self.locals[-1].add(node.name)
         self.locals.append(set())
         try:
             return ASTTransformer.visitClass(self, node)
@@ -681,6 +730,8 @@
             self.locals.pop()
 
     def visitFunction(self, node):
+        if len(self.locals) > 1:
+            self.locals[-1].add(node.name)
         self.locals.append(set(node.argnames))
         try:
             return ASTTransformer.visitFunction(self, node)
@@ -711,12 +762,11 @@
     def visitName(self, node):
         # If the name refers to a local inside a lambda, list comprehension, or
         # generator expression, leave it alone
-        for frame in self.locals:
-            if node.name in frame:
-                return node
-        # Otherwise, translate the name ref into a context lookup
-        func_args = [ast.Name('data'), ast.Const(node.name)]
-        return ast.CallFunc(ast.Name('_lookup_name'), func_args)
+        if node.name not in flatten(self.locals):
+            # Otherwise, translate the name ref into a context lookup
+            func_args = [ast.Name('__data__'), ast.Const(node.name)]
+            node = ast.CallFunc(ast.Name('_lookup_name'), func_args)
+        return node
 
 
 class ExpressionASTTransformer(TemplateASTTransformer):
--- a/genshi/template/interpolation.py
+++ b/genshi/template/interpolation.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2007 Edgewall Software
+# Copyright (C) 2007-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -30,8 +30,7 @@
 NAMECHARS = NAMESTART + '.0123456789'
 PREFIX = '$'
 
-def interpolate(text, basedir=None, filename=None, lineno=-1, offset=0,
-                lookup='lenient'):
+def interpolate(text, filepath=None, lineno=-1, offset=0, lookup='strict'):
     """Parse the given string and extract expressions.
     
     This function is a generator that yields `TEXT` events for literal strings,
@@ -45,9 +44,8 @@
     TEXT u'bar'
     
     :param text: the text to parse
-    :param basedir: base directory of the file in which the text was found
-                    (optional)
-    :param filename: basename of the file in which the text was found (optional)
+    :param filepath: absolute path to the file in which the text was found
+                     (optional)
     :param lineno: the line number at which the text was found (optional)
     :param offset: the column number at which the text starts in the source
                    (optional)
@@ -57,9 +55,6 @@
     :raise TemplateSyntaxError: when a syntax error in an expression is
                                 encountered
     """
-    filepath = filename
-    if filepath and basedir:
-        filepath = os.path.join(basedir, filepath)
     pos = [filepath, lineno, offset]
 
     textbuf = []
@@ -73,7 +68,7 @@
             if chunk:
                 try:
                     expr = Expression(chunk.strip(), pos[0], pos[1],
-                                     lookup=lookup)
+                                      lookup=lookup)
                     yield EXPR, expr, tuple(pos)
                 except SyntaxError, err:
                     raise TemplateSyntaxError(err, filepath, pos[1],
--- a/genshi/template/loader.py
+++ b/genshi/template/loader.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -77,12 +77,15 @@
     """
     def __init__(self, search_path=None, auto_reload=False,
                  default_encoding=None, max_cache_size=25, default_class=None,
-                 variable_lookup='lenient', allow_exec=True, callback=None):
+                 variable_lookup='strict', allow_exec=True, callback=None):
         """Create the template laoder.
         
         :param search_path: a list of absolute path names that should be
                             searched for template files, or a string containing
-                            a single absolute path
+                            a single absolute path; alternatively, any item on
+                            the list may be a ''load function'' that is passed
+                            a filename and returns a file-like object and some
+                            metadata
         :param auto_reload: whether to check the last modification time of
                             template files, and reload them if they have changed
         :param default_encoding: the default encoding to assume when loading
@@ -91,8 +94,8 @@
                                cache
         :param default_class: the default `Template` subclass to use when
                               instantiating templates
-        :param variable_lookup: the variable lookup mechanism; either "lenient"
-                                (the default), "strict", or a custom lookup
+        :param variable_lookup: the variable lookup mechanism; either "strict"
+                                (the default), "lenient", or a custom lookup
                                 class
         :param allow_exec: whether to allow Python code blocks in templates
         :param callback: (optional) a callback function that is invoked after a
@@ -109,7 +112,7 @@
         self.search_path = search_path
         if self.search_path is None:
             self.search_path = []
-        elif isinstance(self.search_path, basestring):
+        elif not isinstance(self.search_path, (list, tuple)):
             self.search_path = [self.search_path]
 
         self.auto_reload = auto_reload
@@ -124,15 +127,15 @@
             raise TypeError('The "callback" parameter needs to be callable')
         self.callback = callback
         self._cache = LRUCache(max_cache_size)
-        self._mtime = {}
+        self._uptodate = {}
         self._lock = threading.RLock()
 
     def load(self, filename, relative_to=None, cls=None, encoding=None):
         """Load the template with the given name.
         
-        If the `filename` parameter is relative, this method searches the search
-        path trying to locate a template matching the given name. If the file
-        name is an absolute path, the search path is ignored.
+        If the `filename` parameter is relative, this method searches the
+        search path trying to locate a template matching the given name. If the
+        file name is an absolute path, the search path is ignored.
         
         If the requested template is not found, a `TemplateNotFound` exception
         is raised. Otherwise, a `Template` object is returned that represents
@@ -155,26 +158,27 @@
         :param encoding: the encoding of the template to load; defaults to the
                          ``default_encoding`` of the loader instance
         :return: the loaded `Template` instance
-        :raises TemplateNotFound: if a template with the given name could not be
-                                  found
+        :raises TemplateNotFound: if a template with the given name could not
+                                  be found
         """
         if cls is None:
             cls = self.default_class
-        if encoding is None:
-            encoding = self.default_encoding
         if relative_to and not os.path.isabs(relative_to):
             filename = os.path.join(os.path.dirname(relative_to), filename)
         filename = os.path.normpath(filename)
+        cachekey = filename
 
         self._lock.acquire()
         try:
             # First check the cache to avoid reparsing the same file
             try:
-                tmpl = self._cache[filename]
-                if not self.auto_reload or \
-                        os.path.getmtime(tmpl.filepath) == self._mtime[filename]:
+                tmpl = self._cache[cachekey]
+                if not self.auto_reload:
                     return tmpl
-            except KeyError:
+                uptodate = self._uptodate[cachekey]
+                if uptodate is not None and uptodate():
+                    return tmpl
+            except KeyError, OSError:
                 pass
 
             search_path = self.search_path
@@ -190,41 +194,135 @@
                 # template is on the search path
                 dirname = os.path.dirname(relative_to)
                 if dirname not in search_path:
-                    search_path = search_path + [dirname]
+                    search_path = list(search_path) + [dirname]
                 isabs = True
 
             elif not search_path:
                 # Uh oh, don't know where to look for the template
                 raise TemplateError('Search path for templates not configured')
 
-            for dirname in search_path:
-                filepath = os.path.join(dirname, filename)
+            for loadfunc in search_path:
+                if isinstance(loadfunc, basestring):
+                    loadfunc = directory(loadfunc)
                 try:
-                    fileobj = open(filepath, 'U')
+                    filepath, filename, fileobj, uptodate = loadfunc(filename)
+                except IOError:
+                    continue
+                else:
                     try:
                         if isabs:
                             # If the filename of either the included or the 
                             # including template is absolute, make sure the
                             # included template gets an absolute path, too,
-                            # so that nested include work properly without a
+                            # so that nested includes work properly without a
                             # search path
-                            filename = os.path.join(dirname, filename)
-                            dirname = ''
-                        tmpl = cls(fileobj, basedir=dirname, filename=filename,
-                                   loader=self, encoding=encoding,
-                                   lookup=self.variable_lookup,
-                                   allow_exec=self.allow_exec)
+                            filename = filepath
+                        tmpl = self._instantiate(cls, fileobj, filepath,
+                                                 filename, encoding=encoding)
                         if self.callback:
                             self.callback(tmpl)
-                        self._cache[filename] = tmpl
-                        self._mtime[filename] = os.path.getmtime(filepath)
+                        self._cache[cachekey] = tmpl
+                        self._uptodate[cachekey] = uptodate
                     finally:
-                        fileobj.close()
+                        if hasattr(fileobj, 'close'):
+                            fileobj.close()
                     return tmpl
-                except IOError:
-                    continue
 
             raise TemplateNotFound(filename, search_path)
 
         finally:
             self._lock.release()
+
+    def _instantiate(self, cls, fileobj, filepath, filename, encoding=None):
+        """Instantiate and return the `Template` object based on the given
+        class and parameters.
+        
+        This function is intended for subclasses to override if they need to
+        implement special template instantiation logic. Code that just uses
+        the `TemplateLoader` should use the `load` method instead.
+        
+        :param cls: the class of the template object to instantiate
+        :param fileobj: a readable file-like object containing the template
+                        source
+        :param filepath: the absolute path to the template file
+        :param filename: the path to the template file relative to the search
+                         path
+        :param encoding: the encoding of the template to load; defaults to the
+                         ``default_encoding`` of the loader instance
+        :return: the loaded `Template` instance
+        :rtype: `Template`
+        """
+        if encoding is None:
+            encoding = self.default_encoding
+        return cls(fileobj, filepath=filepath, filename=filename, loader=self,
+                   encoding=encoding, lookup=self.variable_lookup,
+                   allow_exec=self.allow_exec)
+
+    def directory(path):
+        """Loader factory for loading templates from a local directory.
+        
+        :param path: the path to the local directory containing the templates
+        :return: the loader function to load templates from the given directory
+        :rtype: ``function``
+        """
+        def _load_from_directory(filename):
+            filepath = os.path.join(path, filename)
+            fileobj = open(filepath, 'U')
+            mtime = os.path.getmtime(filepath)
+            def _uptodate():
+                return mtime == os.path.getmtime(filepath)
+            return filepath, filename, fileobj, _uptodate
+        return _load_from_directory
+    directory = staticmethod(directory)
+
+    def package(name, path):
+        """Loader factory for loading templates from egg package data.
+        
+        :param name: the name of the package containing the resources
+        :param path: the path inside the package data
+        :return: the loader function to load templates from the given package
+        :rtype: ``function``
+        """
+        from pkg_resources import resource_stream
+        def _load_from_package(filename):
+            filepath = os.path.join(path, filename)
+            return filepath, filename, resource_stream(name, filepath), None
+        return _load_from_package
+    package = staticmethod(package)
+
+    def prefixed(**delegates):
+        """Factory for a load function that delegates to other loaders
+        depending on the prefix of the requested template path.
+        
+        The prefix is stripped from the filename when passing on the load
+        request to the delegate.
+        
+        >>> load = prefixed(
+        ...     app1 = lambda filename: ('app1', filename, None, None),
+        ...     app2 = lambda filename: ('app2', filename, None, None)
+        ... )
+        >>> print load('app1/foo.html')
+        ('app1', 'app1/foo.html', None, None)
+        >>> print load('app2/bar.html')
+        ('app2', 'app2/bar.html', None, None)
+        
+        :param delegates: mapping of path prefixes to loader functions
+        :return: the loader function
+        :rtype: ``function``
+        """
+        def _dispatch_by_prefix(filename):
+            for prefix, delegate in delegates.items():
+                if filename.startswith(prefix):
+                    if isinstance(delegate, basestring):
+                        delegate = directory(delegate)
+                    filepath, _, fileobj, uptodate = delegate(
+                        filename[len(prefix):].lstrip('/\\')
+                    )
+                    return filepath, filename, fileobj, uptodate
+            raise TemplateNotFound(filename, delegates.keys())
+        return _dispatch_by_prefix
+    prefixed = staticmethod(prefixed)
+
+directory = TemplateLoader.directory
+package = TemplateLoader.package
+prefixed = TemplateLoader.prefixed
--- a/genshi/template/markup.py
+++ b/genshi/template/markup.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -14,23 +14,17 @@
 """Markup templating engine."""
 
 from itertools import chain
-import sys
-from textwrap import dedent
 
-from genshi.core import Attrs, Namespace, Stream, StreamEventKind
+from genshi.core import Attrs, Markup, Namespace, Stream, StreamEventKind
 from genshi.core import START, END, START_NS, END_NS, TEXT, PI, COMMENT
 from genshi.input import XMLParser
 from genshi.template.base import BadDirectiveError, Template, \
                                  TemplateSyntaxError, _apply_directives, \
-                                 INCLUDE, SUB
+                                 EXEC, INCLUDE, SUB
 from genshi.template.eval import Suite
 from genshi.template.interpolation import interpolate
 from genshi.template.directives import *
-
-if sys.version_info < (2, 4):
-    _ctxt2dict = lambda ctxt: ctxt.frames[0]
-else:
-    _ctxt2dict = lambda ctxt: ctxt
+from genshi.template.text import NewTextTemplate
 
 __all__ = ['MarkupTemplate']
 __docformat__ = 'restructuredtext en'
@@ -47,8 +41,6 @@
       <li>1</li><li>2</li><li>3</li>
     </ul>
     """
-    EXEC = StreamEventKind('EXEC')
-    """Stream event kind representing a Python code suite to execute."""
 
     DIRECTIVE_NAMESPACE = Namespace('http://genshi.edgewall.org/')
     XINCLUDE_NAMESPACE = Namespace('http://www.w3.org/2001/XInclude')
@@ -65,17 +57,16 @@
                   ('content', ContentDirective),
                   ('attrs', AttrsDirective),
                   ('strip', StripDirective)]
+    serializer = 'xml'
+    _number_conv = Markup
 
-    def __init__(self, source, basedir=None, filename=None, loader=None,
-                 encoding=None, lookup='lenient', allow_exec=True):
-        Template.__init__(self, source, basedir=basedir, filename=filename,
-                          loader=loader, encoding=encoding, lookup=lookup,
-                          allow_exec=allow_exec)
+    def _init_filters(self):
+        Template._init_filters(self)
         # Make sure the include filter comes after the match filter
-        if loader:
+        if self.loader:
             self.filters.remove(self._include)
-        self.filters += [self._exec, self._match]
-        if loader:
+        self.filters += [self._match]
+        if self.loader:
             self.filters.append(self._include)
 
     def _parse(self, source, encoding):
@@ -83,8 +74,8 @@
         dirmap = {} # temporary mapping of directives to elements
         ns_prefix = {}
         depth = 0
-        in_fallback = 0
-        include_href = None
+        fallbacks = []
+        includes = []
 
         if not isinstance(source, Stream):
             source = XMLParser(source, filename=self.filename,
@@ -133,8 +124,8 @@
                         directives.append((cls, value, ns_prefix.copy(), pos))
                     else:
                         if value:
-                            value = list(interpolate(value, self.basedir,
-                                                     pos[0], pos[1], pos[2],
+                            value = list(interpolate(value, self.filepath,
+                                                     pos[1], pos[2],
                                                      lookup=self.lookup))
                             if len(value) == 1 and value[0][0] is TEXT:
                                 value = value[0][1]
@@ -155,9 +146,11 @@
                             raise TemplateSyntaxError('Include misses required '
                                                       'attribute "href"',
                                                       self.filepath, *pos[1:])
+                        includes.append((include_href, new_attrs.get('parse')))
                         streams.append([])
                     elif tag.localname == 'fallback':
-                        in_fallback += 1
+                        streams.append([])
+                        fallbacks.append(streams[-1])
 
                 else:
                     stream.append((kind, (tag, new_attrs), pos))
@@ -167,12 +160,26 @@
             elif kind is END:
                 depth -= 1
 
-                if in_fallback and data == self.XINCLUDE_NAMESPACE['fallback']:
-                    in_fallback -= 1
+                if fallbacks and data == self.XINCLUDE_NAMESPACE['fallback']:
+                    assert streams.pop() is fallbacks[-1]
                 elif data == self.XINCLUDE_NAMESPACE['include']:
-                    fallback = streams.pop()
+                    fallback = None
+                    if len(fallbacks) == len(includes):
+                        fallback = fallbacks.pop()
+                    streams.pop() # discard anything between the include tags
+                                  # and the fallback element
                     stream = streams[-1]
-                    stream.append((INCLUDE, (include_href, fallback), pos))
+                    href, parse = includes.pop()
+                    try:
+                        cls = {
+                            'xml': MarkupTemplate,
+                            'text': NewTextTemplate
+                        }[parse or 'xml']
+                    except KeyError:
+                        raise TemplateSyntaxError('Invalid value for "parse" '
+                                                  'attribute of include',
+                                                  self.filepath, *pos[1:])
+                    stream.append((INCLUDE, (href, cls, fallback), pos))
                 else:
                     stream.append((kind, data, pos))
 
@@ -191,19 +198,7 @@
                     raise TemplateSyntaxError('Python code blocks not allowed',
                                               self.filepath, *pos[1:])
                 try:
-                    # As Expat doesn't report whitespace between the PI target
-                    # and the data, we have to jump through some hoops here to
-                    # get correctly indented Python code
-                    # Unfortunately, we'll still probably not get the line
-                    # number quite right
-                    lines = [line.expandtabs() for line in data[1].splitlines()]
-                    first = lines[0]
-                    rest = dedent('\n'.join(lines[1:])).rstrip()
-                    if first.rstrip().endswith(':') and not rest[0].isspace():
-                        rest = '\n'.join(['    ' + line for line
-                                          in rest.splitlines()])
-                    source = '\n'.join([first, rest])
-                    suite = Suite(source, self.filepath, pos[1],
+                    suite = Suite(data[1], self.filepath, pos[1],
                                   lookup=self.lookup)
                 except SyntaxError, err:
                     raise TemplateSyntaxError(err, self.filepath,
@@ -212,9 +207,8 @@
                 stream.append((EXEC, suite, pos))
 
             elif kind is TEXT:
-                for kind, data, pos in interpolate(data, self.basedir, pos[0],
-                                                   pos[1], pos[2],
-                                                   lookup=self.lookup):
+                for kind, data, pos in interpolate(data, self.filepath, pos[1],
+                                                   pos[2], lookup=self.lookup):
                     stream.append((kind, data, pos))
 
             elif kind is COMMENT:
@@ -227,17 +221,7 @@
         assert len(streams) == 1
         return streams[0]
 
-    def _exec(self, stream, ctxt):
-        """Internal stream filter that executes code in ``<?python ?>``
-        processing instructions.
-        """
-        for event in stream:
-            if event[0] is EXEC:
-                event[1].execute(ctxt.data)
-            else:
-                yield event
-
-    def _match(self, stream, ctxt, match_templates=None):
+    def _match(self, stream, ctxt, match_templates=None, **vars):
         """Internal stream filter that applies any defined match templates
         to the stream.
         """
@@ -268,10 +252,13 @@
                 yield event
                 continue
 
-            for idx, (test, path, template, namespaces, directives) in \
-                    enumerate(match_templates):
+            for idx, (test, path, template, hints, namespaces, directives) \
+                    in enumerate(match_templates):
 
                 if test(event, namespaces, ctxt) is True:
+                    if 'match_once' in hints:
+                        del match_templates[idx]
+                        idx -= 1
 
                     # Let the remaining match templates know about the event so
                     # they get a chance to update their internal state
@@ -280,35 +267,39 @@
 
                     # Consume and store all events until an end event
                     # corresponding to this start event is encountered
-                    content = chain([event],
-                                    self._match(_strip(stream), ctxt,
-                                                [match_templates[idx]]),
-                                    tail)
-                    content = list(self._include(content, ctxt))
+                    pre_match_templates = match_templates[:idx + 1]
+                    if 'match_once' not in hints and 'not_recursive' in hints:
+                        pre_match_templates.pop()
+                    inner = _strip(stream)
+                    if pre_match_templates:
+                        inner = self._match(inner, ctxt, pre_match_templates)
+                    content = self._include(chain([event], inner, tail), ctxt)
+                    if 'not_buffered' not in hints:
+                        content = list(content)
 
-                    for test in [mt[0] for mt in match_templates]:
-                        test(tail[0], namespaces, ctxt, updateonly=True)
+                    if tail:
+                        for test in [mt[0] for mt in match_templates]:
+                            test(tail[0], namespaces, ctxt, updateonly=True)
 
                     # Make the select() function available in the body of the
                     # match template
                     def select(path):
                         return Stream(content).select(path, namespaces, ctxt)
-                    ctxt.push(dict(select=select))
+                    vars = dict(select=select)
 
                     # Recursively process the output
-                    template = _apply_directives(template, ctxt, directives)
-                    for event in self._match(self._eval(self._flatten(template,
-                                                                      ctxt),
-                                                        ctxt), ctxt,
-                                             match_templates[:idx] +
-                                             match_templates[idx + 1:]):
+                    template = _apply_directives(template, directives, ctxt,
+                                                 **vars)
+                    for event in self._match(
+                            self._exec(
+                                self._eval(
+                                    self._flatten(template, ctxt, **vars),
+                                    ctxt, **vars),
+                                ctxt, **vars),
+                            ctxt, match_templates[idx + 1:], **vars):
                         yield event
 
-                    ctxt.pop()
                     break
 
             else: # no matches
                 yield event
-
-
-EXEC = MarkupTemplate.EXEC
--- a/genshi/template/plugin.py
+++ b/genshi/template/plugin.py
@@ -23,7 +23,7 @@
 from genshi.template.base import Template
 from genshi.template.loader import TemplateLoader
 from genshi.template.markup import MarkupTemplate
-from genshi.template.text import TextTemplate
+from genshi.template.text import TextTemplate, NewTextTemplate
 
 __all__ = ['ConfigurationError', 'AbstractTemplateEnginePlugin',
            'MarkupTemplateEnginePlugin', 'TextTemplateEnginePlugin']
@@ -62,7 +62,7 @@
         if loader_callback and not callable(loader_callback):
             raise ConfigurationError('loader callback must be a function')
 
-        lookup_errors = options.get('genshi.lookup_errors', 'lenient')
+        lookup_errors = options.get('genshi.lookup_errors', 'strict')
         if lookup_errors not in ('lenient', 'strict'):
             raise ConfigurationError('Unknown lookup errors mode "%s"' %
                                      lookup_errors)
@@ -97,7 +97,7 @@
 
         return self.loader.load(templatename)
 
-    def _get_render_options(self, format=None):
+    def _get_render_options(self, format=None, fragment=False):
         if format is None:
             format = self.default_format
         kwargs = {'method': format}
@@ -107,7 +107,7 @@
 
     def render(self, info, format=None, fragment=False, template=None):
         """Render the template to a string using the provided info."""
-        kwargs = self._get_render_options(format=format)
+        kwargs = self._get_render_options(format=format, fragment=fragment)
         return self.transform(info, template).render(**kwargs)
 
     def transform(self, info, template):
@@ -140,10 +140,10 @@
             raise ConfigurationError('Unknown output format %r' % format)
         self.default_format = format
 
-    def _get_render_options(self, format=None):
+    def _get_render_options(self, format=None, fragment=False):
         kwargs = super(MarkupTemplateEnginePlugin,
-                       self)._get_render_options(format)
-        if self.default_doctype:
+                       self)._get_render_options(format, fragment)
+        if self.default_doctype and not fragment:
             kwargs['doctype'] = self.default_doctype
         return kwargs
 
@@ -162,3 +162,15 @@
     template_class = TextTemplate
     extension = '.txt'
     default_format = 'text'
+
+    def __init__(self, extra_vars_func=None, options=None):
+        if options is None:
+            options = {}
+
+        new_syntax = options.get('genshi.new_text_syntax')
+        if isinstance(new_syntax, basestring):
+            new_syntax = new_syntax.lower() in ('1', 'on', 'yes', 'true')
+        if new_syntax:
+            self.template_class = NewTextTemplate
+
+        AbstractTemplateEnginePlugin.__init__(self, extra_vars_func, options)
--- a/genshi/template/tests/directives.py
+++ b/genshi/template/tests/directives.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -382,6 +382,7 @@
         """)
         self.assertEqual("""
                       Hi, you!
+
         """, str(tmpl.generate()))
 
     def test_function_with_star_args(self):
@@ -482,9 +483,36 @@
         try:
             list(tmpl.generate(foo=12))
             self.fail('Expected TemplateRuntimeError')
-        except TemplateRuntimeError, e:
+        except TypeError, e:
+            assert (str(e) == "iteration over non-sequence" or
+                    str(e) == "'int' object is not iterable")
+            exc_type, exc_value, exc_traceback = sys.exc_info()
+            frame = exc_traceback.tb_next
+            frames = []
+            while frame.tb_next:
+                frame = frame.tb_next
+                frames.append(frame)
+            self.assertEqual("<Expression u'iter(foo)'>",
+                             frames[-1].tb_frame.f_code.co_name)
+            self.assertEqual('test.html',
+                             frames[-1].tb_frame.f_code.co_filename)
+            if sys.version_info[:2] >= (2, 4):
+                self.assertEqual(2, frames[-1].tb_lineno)
+
+    def test_for_with_empty_value(self):
+        """
+        Verify an empty 'for' value is an error
+        """
+        try:
+            MarkupTemplate("""<doc xmlns:py="http://genshi.edgewall.org/">
+          <py:for each="">
+            empty
+          </py:for>
+        </doc>""", filename='test.html')
+            self.fail('ExpectedTemplateSyntaxError')
+        except TemplateSyntaxError, e:
             self.assertEqual('test.html', e.filename)
-            if sys.version_info[:2] >= (2, 4):
+            if sys.version_info[:2] > (2,4):
                 self.assertEqual(2, e.lineno)
 
 
@@ -620,6 +648,32 @@
           </body>
         </html>""", str(tmpl.generate()))
 
+    def test_recursive_match_3(self):
+        tmpl = MarkupTemplate("""<test xmlns:py="http://genshi.edgewall.org/">
+          <py:match path="b[@type='bullet']">
+            <bullet>${select('*|text()')}</bullet>
+          </py:match>
+          <py:match path="group[@type='bullet']">
+            <ul>${select('*')}</ul>
+          </py:match>
+          <py:match path="b">
+            <generic>${select('*|text()')}</generic>
+          </py:match>
+
+          <b>
+            <group type="bullet">
+              <b type="bullet">1</b>
+              <b type="bullet">2</b>
+            </group>
+          </b>
+        </test>
+        """)
+        self.assertEqual("""<test>
+            <generic>
+            <ul><bullet>1</bullet><bullet>2</bullet></ul>
+          </generic>
+        </test>""", str(tmpl.generate()))
+
     def test_not_match_self(self):
         """
         See http://genshi.edgewall.org/ticket/77
@@ -842,6 +896,54 @@
           </body>
         </html>""", str(tmpl.generate()))
 
+    def test_match_with_once_attribute(self):
+        tmpl = MarkupTemplate("""<html xmlns:py="http://genshi.edgewall.org/">
+          <py:match path="body" once="true"><body>
+            <div id="wrap">
+              ${select("*")}
+            </div>
+          </body></py:match>
+          <body>
+            <p>Foo</p>
+          </body>
+          <body>
+            <p>Bar</p>
+          </body>
+        </html>""")
+        self.assertEqual("""<html>
+          <body>
+            <div id="wrap">
+              <p>Foo</p>
+            </div>
+          </body>
+          <body>
+            <p>Bar</p>
+          </body>
+        </html>""", str(tmpl.generate()))
+
+    def test_match_with_recursive_attribute(self):
+        tmpl = MarkupTemplate("""<doc xmlns:py="http://genshi.edgewall.org/">
+          <py:match path="elem" recursive="false"><elem>
+            <div class="elem">
+              ${select('*')}
+            </div>
+          </elem></py:match>
+          <elem>
+            <subelem>
+              <elem/>
+            </subelem>
+          </elem>
+        </doc>""")
+        self.assertEqual("""<doc>
+          <elem>
+            <div class="elem">
+              <subelem>
+              <elem/>
+            </subelem>
+            </div>
+          </elem>
+        </doc>""", str(tmpl.generate()))
+
     # FIXME
     #def test_match_after_step(self):
     #    tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/">
@@ -857,6 +959,21 @@
     #    </div>""", str(tmpl.generate()))
 
 
+class ContentDirectiveTestCase(unittest.TestCase):
+    """Tests for the `py:content` template directive."""
+
+    def test_as_element(self):
+        try:
+            tmpl = MarkupTemplate("""<doc xmlns:py="http://genshi.edgewall.org/">
+              <py:content foo="">Foo</py:content>
+            </doc>""", filename='test.html')
+            self.fail('Expected TemplateSyntaxError')
+        except TemplateSyntaxError, e:
+            self.assertEqual('test.html', e.filename)
+            if sys.version_info[:2] >= (2, 4):
+                self.assertEqual(2, e.lineno)
+
+
 class ReplaceDirectiveTestCase(unittest.TestCase):
     """Tests for the `py:replace` template directive."""
 
@@ -875,6 +992,14 @@
             if sys.version_info[:2] >= (2, 4):
                 self.assertEqual(2, e.lineno)
 
+    def test_as_element(self):
+        tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/">
+          <py:replace value="title" />
+        </div>""", filename='test.html')
+        self.assertEqual("""<div>
+          Test
+        </div>""", str(tmpl.generate(title='Test')))
+
 
 class StripDirectiveTestCase(unittest.TestCase):
     """Tests for the `py:strip` template directive."""
@@ -968,6 +1093,22 @@
             here are two semicolons: ;;
         </div>""", str(tmpl.generate()))
 
+    def test_ast_transformation(self):
+        """
+        Verify that the usual template expression AST transformations are
+        applied despite the code being compiled to a `Suite` object.
+        """
+        tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/">
+          <span py:with="bar=foo.bar">
+            $bar
+          </span>
+        </div>""")
+        self.assertEqual("""<div>
+          <span>
+            42
+          </span>
+        </div>""", str(tmpl.generate(foo={'bar': 42})))
+
     def test_unicode_expr(self):
         tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/">
           <span py:with="weeks=(u'一', u'二', u'三', u'四', u'五', u'六', u'日')">
@@ -979,6 +1120,16 @@
             一二三四五六日
           </span>
         </div>""", str(tmpl.generate()))
+        
+    def test_with_empty_value(self):
+        """
+        Verify that an empty py:with works (useless, but legal)
+        """
+        tmpl = MarkupTemplate("""<div xmlns:py="http://genshi.edgewall.org/">
+          <span py:with="">Text</span></div>""")
+
+        self.assertEqual("""<div>
+          <span>Text</span></div>""", str(tmpl.generate()))
 
 
 def suite():
@@ -990,6 +1141,7 @@
     suite.addTest(unittest.makeSuite(ForDirectiveTestCase, 'test'))
     suite.addTest(unittest.makeSuite(IfDirectiveTestCase, 'test'))
     suite.addTest(unittest.makeSuite(MatchDirectiveTestCase, 'test'))
+    suite.addTest(unittest.makeSuite(ContentDirectiveTestCase, 'test'))
     suite.addTest(unittest.makeSuite(ReplaceDirectiveTestCase, 'test'))
     suite.addTest(unittest.makeSuite(StripDirectiveTestCase, 'test'))
     suite.addTest(unittest.makeSuite(WithDirectiveTestCase, 'test'))
--- a/genshi/template/tests/eval.py
+++ b/genshi/template/tests/eval.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -12,6 +12,8 @@
 # history and logs, available at http://genshi.edgewall.org/log/.
 
 import doctest
+import pickle
+from StringIO import StringIO
 import sys
 import unittest
 
@@ -32,6 +34,14 @@
         self.assertEqual(hash(expr), hash(Expression('x,y')))
         self.assertNotEqual(hash(expr), hash(Expression('y, x')))
 
+    def test_pickle(self):
+        expr = Expression('1 < 2')
+        buf = StringIO()
+        pickle.dump(expr, buf, 2)
+        buf.seek(0)
+        unpickled = pickle.load(buf)
+        assert unpickled.evaluate({}) is True
+
     def test_name_lookup(self):
         self.assertEqual('bar', Expression('foo').evaluate({'foo': 'bar'}))
         self.assertEqual(id, Expression('id').evaluate({}))
@@ -321,7 +331,8 @@
         self.assertEqual([0, 1, 2, 3], expr.evaluate({'numbers': range(5)}))
 
     def test_access_undefined(self):
-        expr = Expression("nothing", filename='index.html', lineno=50)
+        expr = Expression("nothing", filename='index.html', lineno=50,
+                          lookup='lenient')
         retval = expr.evaluate({})
         assert isinstance(retval, Undefined)
         self.assertEqual('nothing', retval._name)
@@ -332,23 +343,45 @@
             def __repr__(self):
                 return '<Something>'
         something = Something()
-        expr = Expression('something.nil', filename='index.html', lineno=50)
+        expr = Expression('something.nil', filename='index.html', lineno=50,
+                          lookup='lenient')
         retval = expr.evaluate({'something': something})
         assert isinstance(retval, Undefined)
         self.assertEqual('nil', retval._name)
         assert retval._owner is something
 
+    def test_getattr_exception(self):
+        class Something(object):
+            def prop_a(self):
+                raise NotImplementedError
+            prop_a = property(prop_a)
+            def prop_b(self):
+                raise AttributeError
+            prop_b = property(prop_b)
+        self.assertRaises(NotImplementedError,
+                          Expression('s.prop_a').evaluate, {'s': Something()})
+        self.assertRaises(AttributeError,
+                          Expression('s.prop_b').evaluate, {'s': Something()})
+
     def test_getitem_undefined_string(self):
         class Something(object):
             def __repr__(self):
                 return '<Something>'
         something = Something()
-        expr = Expression('something["nil"]', filename='index.html', lineno=50)
+        expr = Expression('something["nil"]', filename='index.html', lineno=50,
+                          lookup='lenient')
         retval = expr.evaluate({'something': something})
         assert isinstance(retval, Undefined)
         self.assertEqual('nil', retval._name)
         assert retval._owner is something
 
+    def test_getitem_exception(self):
+        class Something(object):
+            def __getitem__(self, key):
+                raise NotImplementedError
+        self.assertRaises(NotImplementedError,
+                          Expression('s["foo"]').evaluate, {'s': Something()})
+
     def test_error_access_undefined(self):
         expr = Expression("nothing", filename='index.html', lineno=50,
                           lookup='strict')
@@ -420,6 +453,29 @@
 
 class SuiteTestCase(unittest.TestCase):
 
+    def test_pickle(self):
+        suite = Suite('foo = 42')
+        buf = StringIO()
+        pickle.dump(suite, buf, 2)
+        buf.seek(0)
+        unpickled = pickle.load(buf)
+        data = {}
+        unpickled.execute(data)
+        self.assertEqual(42, data['foo'])
+
+    def test_internal_shadowing(self):
+        # The context itself is stored in the global execution scope of a suite
+        # It used to get stored under the name 'data', which meant the
+        # following test would fail, as the user defined 'data' variable
+        # shadowed the Genshi one. We now use the name '__data__' to avoid
+        # conflicts
+        suite = Suite("""data = []
+bar = foo
+""")
+        data = {'foo': 42}
+        suite.execute(data)
+        self.assertEqual(42, data['bar'])
+
     def test_assign(self):
         suite = Suite("foo = 42")
         data = {}
@@ -434,7 +490,8 @@
         self.assertEqual(None, data['donothing']())
 
     def test_def_with_multiple_statements(self):
-        suite = Suite("""def donothing():
+        suite = Suite("""
+def donothing():
     if True:
         return foo
 """)
@@ -443,6 +500,35 @@
         assert 'donothing' in data
         self.assertEqual('bar', data['donothing']())
 
+    def test_def_using_nonlocal(self):
+        suite = Suite("""
+values = []
+def add(value):
+    if value not in values:
+        values.append(value)
+add('foo')
+add('bar')
+""")
+        data = {}
+        suite.execute(data)
+        self.assertEqual(['foo', 'bar'], data['values'])
+
+    def test_def_nested(self):
+        suite = Suite("""
+def doit():
+    values = []
+    def add(value):
+        if value not in values:
+            values.append(value)
+    add('foo')
+    add('bar')
+    return values
+x = doit()
+""")
+        data = {}
+        suite.execute(data)
+        self.assertEqual(['foo', 'bar'], data['x'])
+
     def test_delete(self):
         suite = Suite("""foo = 42
 del foo
@@ -457,6 +543,28 @@
         suite.execute(data)
         assert 'plain' in data
 
+    def test_class_in_def(self):
+        suite = Suite("""
+def create():
+    class Foobar(object):
+        def __str__(self):
+            return 'foobar'
+    return Foobar()
+x = create()
+""")
+        data = {}
+        suite.execute(data)
+        self.assertEqual('foobar', str(data['x']))
+
+    def test_class_with_methods(self):
+        suite = Suite("""class plain(object):
+    def donothing():
+        pass
+""")
+        data = {}
+        suite.execute(data)
+        assert 'plain' in data
+
     def test_import(self):
         suite = Suite("from itertools import ifilter")
         data = {}
@@ -525,6 +633,25 @@
     def test_local_augmented_assign(self):
         Suite("x = 1; x += 42; assert x == 43").execute({})
 
+    def test_augmented_assign_in_def(self):
+        d = {}
+        Suite("""def foo():
+    i = 1
+    i += 1
+    return i
+x = foo()""").execute(d)
+        self.assertEqual(2, d['x'])
+
+    def test_augmented_assign_in_loop_in_def(self):
+        d = {}
+        Suite("""def foo():
+    i = 0
+    for n in range(5):
+        i += n
+    return i
+x = foo()""").execute(d)
+        self.assertEqual(10, d['x'])
+
     def test_assign_in_list(self):
         suite = Suite("[d['k']] = 'foo',; assert d['k'] == 'foo'")
         d = {"k": "bar"}
--- a/genshi/template/tests/interpolation.py
+++ b/genshi/template/tests/interpolation.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2007 Edgewall Software
+# Copyright (C) 2007-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
--- a/genshi/template/tests/loader.py
+++ b/genshi/template/tests/loader.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -104,6 +104,34 @@
               <div>Included</div>
             </html>""", tmpl.generate().render())
 
+    def test_relative_include_samesubdir(self):
+        file1 = open(os.path.join(self.dirname, 'tmpl1.html'), 'w')
+        try:
+            file1.write("""<div>Included tmpl1.html</div>""")
+        finally:
+            file1.close()
+
+        os.mkdir(os.path.join(self.dirname, 'sub'))
+        file2 = open(os.path.join(self.dirname, 'sub', 'tmpl1.html'), 'w')
+        try:
+            file2.write("""<div>Included sub/tmpl1.html</div>""")
+        finally:
+            file2.close()
+
+        file3 = open(os.path.join(self.dirname, 'sub', 'tmpl2.html'), 'w')
+        try:
+            file3.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude">
+              <xi:include href="tmpl1.html" />
+            </html>""")
+        finally:
+            file3.close()
+
+        loader = TemplateLoader([self.dirname])
+        tmpl = loader.load('sub/tmpl2.html')
+        self.assertEqual("""<html>
+              <div>Included sub/tmpl1.html</div>
+            </html>""", tmpl.generate().render())
+
     def test_relative_include_without_search_path(self):
         file1 = open(os.path.join(self.dirname, 'tmpl1.html'), 'w')
         try:
@@ -172,6 +200,67 @@
           <div>Included</div>
         </html>""", tmpl2.generate().render())
 
+    def test_relative_absolute_template_preferred(self):
+        file1 = open(os.path.join(self.dirname, 'tmpl1.html'), 'w')
+        try:
+            file1.write("""<div>Included</div>""")
+        finally:
+            file1.close()
+
+        os.mkdir(os.path.join(self.dirname, 'sub'))
+        file2 = open(os.path.join(self.dirname, 'sub', 'tmpl1.html'), 'w')
+        try:
+            file2.write("""<div>Included from sub</div>""")
+        finally:
+            file2.close()
+
+        file3 = open(os.path.join(self.dirname, 'sub', 'tmpl2.html'), 'w')
+        try:
+            file3.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude">
+              <xi:include href="tmpl1.html" />
+            </html>""")
+        finally:
+            file3.close()
+
+        loader = TemplateLoader()
+        tmpl = loader.load(os.path.abspath(os.path.join(self.dirname, 'sub',
+                                                        'tmpl2.html')))
+        self.assertEqual("""<html>
+              <div>Included from sub</div>
+            </html>""", tmpl.generate().render())
+
+    def test_abspath_caching(self):
+        abspath = os.path.join(self.dirname, 'abs')
+        os.mkdir(abspath)
+        file1 = open(os.path.join(abspath, 'tmpl1.html'), 'w')
+        try:
+            file1.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude">
+              <xi:include href="tmpl2.html" />
+            </html>""")
+        finally:
+            file1.close()
+
+        file2 = open(os.path.join(abspath, 'tmpl2.html'), 'w')
+        try:
+            file2.write("""<div>Included from abspath.</div>""")
+        finally:
+            file2.close()
+
+        searchpath = os.path.join(self.dirname, 'searchpath')
+        os.mkdir(searchpath)
+        file3 = open(os.path.join(searchpath, 'tmpl2.html'), 'w')
+        try:
+            file3.write("""<div>Included from searchpath.</div>""")
+        finally:
+            file3.close()
+
+        loader = TemplateLoader(searchpath)
+        tmpl1 = loader.load(os.path.join(abspath, 'tmpl1.html'))
+        self.assertEqual("""<html>
+              <div>Included from searchpath.</div>
+            </html>""", tmpl1.generate().render())
+        assert 'tmpl2.html' in loader._cache
+
     def test_load_with_default_encoding(self):
         f = open(os.path.join(self.dirname, 'tmpl.html'), 'w')
         try:
@@ -219,6 +308,108 @@
               <p>Hello, hello</p>
             </html>""", tmpl.generate().render())
 
+    def test_prefix_delegation_to_directories(self):
+        """
+        Test prefix delegation with the following layout:
+        
+        templates/foo.html
+        sub1/templates/tmpl1.html
+        sub2/templates/tmpl2.html
+        
+        Where sub1 and sub2 are prefixes, and both tmpl1.html and tmpl2.html
+        incldue foo.html.
+        """
+        dir1 = os.path.join(self.dirname, 'templates')
+        os.mkdir(dir1)
+        file1 = open(os.path.join(dir1, 'foo.html'), 'w')
+        try:
+            file1.write("""<div>Included foo</div>""")
+        finally:
+            file1.close()
+
+        dir2 = os.path.join(self.dirname, 'sub1', 'templates')
+        os.makedirs(dir2)
+        file2 = open(os.path.join(dir2, 'tmpl1.html'), 'w')
+        try:
+            file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude">
+              <xi:include href="../foo.html" /> from sub1
+            </html>""")
+        finally:
+            file2.close()
+
+        dir3 = os.path.join(self.dirname, 'sub2', 'templates')
+        os.makedirs(dir3)
+        file3 = open(os.path.join(dir3, 'tmpl2.html'), 'w')
+        try:
+            file3.write("""<div>tmpl2</div>""")
+        finally:
+            file3.close()
+
+        loader = TemplateLoader([dir1, TemplateLoader.prefixed(
+            sub1 = dir2,
+            sub2 = dir3
+        )])
+        tmpl = loader.load('sub1/tmpl1.html')
+        self.assertEqual("""<html>
+              <div>Included foo</div> from sub1
+            </html>""", tmpl.generate().render())
+
+    def test_prefix_delegation_to_directories_with_subdirs(self):
+        """
+        Test prefix delegation with the following layout:
+        
+        templates/foo.html
+        sub1/templates/tmpl1.html
+        sub1/templates/tmpl2.html
+        sub1/templates/bar/tmpl3.html
+        
+        Where sub1 is a prefix, and tmpl1.html includes all the others.
+        """
+        dir1 = os.path.join(self.dirname, 'templates')
+        os.mkdir(dir1)
+        file1 = open(os.path.join(dir1, 'foo.html'), 'w')
+        try:
+            file1.write("""<div>Included foo</div>""")
+        finally:
+            file1.close()
+
+        dir2 = os.path.join(self.dirname, 'sub1', 'templates')
+        os.makedirs(dir2)
+        file2 = open(os.path.join(dir2, 'tmpl1.html'), 'w')
+        try:
+            file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude">
+              <xi:include href="../foo.html" /> from sub1
+              <xi:include href="tmpl2.html" /> from sub1
+              <xi:include href="bar/tmpl3.html" /> from sub1
+            </html>""")
+        finally:
+            file2.close()
+
+        file3 = open(os.path.join(dir2, 'tmpl2.html'), 'w')
+        try:
+            file3.write("""<div>tmpl2</div>""")
+        finally:
+            file3.close()
+
+        dir3 = os.path.join(self.dirname, 'sub1', 'templates', 'bar')
+        os.makedirs(dir3)
+        file4 = open(os.path.join(dir3, 'tmpl3.html'), 'w')
+        try:
+            file4.write("""<div>bar/tmpl3</div>""")
+        finally:
+            file4.close()
+
+        loader = TemplateLoader([dir1, TemplateLoader.prefixed(
+            sub1 = os.path.join(dir2),
+            sub2 = os.path.join(dir3)
+        )])
+        tmpl = loader.load('sub1/tmpl1.html')
+        self.assertEqual("""<html>
+              <div>Included foo</div> from sub1
+              <div>tmpl2</div> from sub1
+              <div>bar/tmpl3</div> from sub1
+            </html>""", tmpl.generate().render())
+
 
 def suite():
     suite = unittest.TestSuite()
--- a/genshi/template/tests/markup.py
+++ b/genshi/template/tests/markup.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -13,6 +13,7 @@
 
 import doctest
 import os
+import pickle
 import shutil
 from StringIO import StringIO
 import sys
@@ -22,7 +23,7 @@
 from genshi.core import Markup
 from genshi.input import XML
 from genshi.template.base import BadDirectiveError, TemplateSyntaxError
-from genshi.template.loader import TemplateLoader
+from genshi.template.loader import TemplateLoader, TemplateNotFound
 from genshi.template.markup import MarkupTemplate
 
 
@@ -39,6 +40,15 @@
         tmpl = MarkupTemplate(stream)
         self.assertEqual('<root> 42 42</root>', str(tmpl.generate(var=42)))
 
+    def test_pickle(self):
+        stream = XML('<root>$var</root>')
+        tmpl = MarkupTemplate(stream)
+        buf = StringIO()
+        pickle.dump(tmpl, buf, 2)
+        buf.seek(0)
+        unpickled = pickle.load(buf)
+        self.assertEqual('<root>42</root>', str(unpickled.generate(var=42)))
+
     def test_interpolate_mixed3(self):
         tmpl = MarkupTemplate('<root> ${var} $var</root>')
         self.assertEqual('<root> 42 42</root>', str(tmpl.generate(var=42)))
@@ -270,7 +280,7 @@
         finally:
             shutil.rmtree(dirname)
 
-    def test_dynamic_inlude_href(self):
+    def test_dynamic_include_href(self):
         dirname = tempfile.mkdtemp(suffix='genshi_test')
         try:
             file1 = open(os.path.join(dirname, 'tmpl1.html'), 'w')
@@ -296,7 +306,7 @@
         finally:
             shutil.rmtree(dirname)
 
-    def test_select_inluded_elements(self):
+    def test_select_included_elements(self):
         dirname = tempfile.mkdtemp(suffix='genshi_test')
         try:
             file1 = open(os.path.join(dirname, 'tmpl1.html'), 'w')
@@ -351,6 +361,23 @@
         finally:
             shutil.rmtree(dirname)
 
+    def test_error_when_include_not_found(self):
+        dirname = tempfile.mkdtemp(suffix='genshi_test')
+        try:
+            file2 = open(os.path.join(dirname, 'tmpl2.html'), 'w')
+            try:
+                file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude">
+                  <xi:include href="tmpl1.html"/>
+                </html>""")
+            finally:
+                file2.close()
+
+            loader = TemplateLoader([dirname], auto_reload=True)
+            tmpl = loader.load('tmpl2.html')
+            self.assertRaises(TemplateNotFound, tmpl.generate().render)
+        finally:
+            shutil.rmtree(dirname)
+
     def test_fallback_when_include_not_found(self):
         dirname = tempfile.mkdtemp(suffix='genshi_test')
         try:
@@ -371,6 +398,26 @@
         finally:
             shutil.rmtree(dirname)
 
+    def test_fallback_when_auto_reload_true(self):
+        dirname = tempfile.mkdtemp(suffix='genshi_test')
+        try:
+            file2 = open(os.path.join(dirname, 'tmpl2.html'), 'w')
+            try:
+                file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude">
+                  <xi:include href="tmpl1.html"><xi:fallback>
+                    Missing</xi:fallback></xi:include>
+                </html>""")
+            finally:
+                file2.close()
+
+            loader = TemplateLoader([dirname], auto_reload=True)
+            tmpl = loader.load('tmpl2.html')
+            self.assertEqual("""<html>
+                    Missing
+                </html>""", tmpl.generate().render())
+        finally:
+            shutil.rmtree(dirname)
+
     def test_include_in_fallback(self):
         dirname = tempfile.mkdtemp(suffix='genshi_test')
         try:
@@ -397,7 +444,7 @@
             loader = TemplateLoader([dirname])
             tmpl = loader.load('tmpl3.html')
             self.assertEqual("""<html>
-                  <div>Included</div>
+                      <div>Included</div>
                 </html>""", tmpl.generate().render())
         finally:
             shutil.rmtree(dirname)
@@ -422,7 +469,36 @@
             loader = TemplateLoader([dirname])
             tmpl = loader.load('tmpl3.html')
             self.assertEqual("""<html>
-                        Missing
+                      Missing
+                </html>""", tmpl.generate().render())
+        finally:
+            shutil.rmtree(dirname)
+
+    def test_nested_include_in_fallback(self):
+        dirname = tempfile.mkdtemp(suffix='genshi_test')
+        try:
+            file1 = open(os.path.join(dirname, 'tmpl2.html'), 'w')
+            try:
+                file1.write("""<div>Included</div>""")
+            finally:
+                file1.close()
+
+            file2 = open(os.path.join(dirname, 'tmpl3.html'), 'w')
+            try:
+                file2.write("""<html xmlns:xi="http://www.w3.org/2001/XInclude">
+                  <xi:include href="tmpl2.html">
+                    <xi:fallback>
+                      <xi:include href="tmpl1.html" />
+                    </xi:fallback>
+                  </xi:include>
+                </html>""")
+            finally:
+                file2.close()
+
+            loader = TemplateLoader([dirname])
+            tmpl = loader.load('tmpl3.html')
+            self.assertEqual("""<html>
+                  <div>Included</div>
                 </html>""", tmpl.generate().render())
         finally:
             shutil.rmtree(dirname)
@@ -530,6 +606,91 @@
         </html>""")
         tmpl = MarkupTemplate(xml, filename='test.html', allow_exec=True)
 
+    def test_exec_in_match(self): 
+        xml = ("""<html xmlns:py="http://genshi.edgewall.org/">
+          <py:match path="body/p">
+            <?python title="wakka wakka wakka" ?>
+            ${title}
+          </py:match>
+          <body><p>moot text</p></body>
+        </html>""")
+        tmpl = MarkupTemplate(xml, filename='test.html', allow_exec=True)
+        self.assertEqual("""<html>
+          <body>
+            wakka wakka wakka
+          </body>
+        </html>""", tmpl.generate().render())
+
+    def test_with_in_match(self): 
+        xml = ("""<html xmlns:py="http://genshi.edgewall.org/">
+          <py:match path="body/p">
+            <h1>${select('text()')}</h1>
+            ${select('.')}
+          </py:match>
+          <body><p py:with="foo='bar'">${foo}</p></body>
+        </html>""")
+        tmpl = MarkupTemplate(xml, filename='test.html')
+        self.assertEqual("""<html>
+          <body>
+            <h1>bar</h1>
+            <p>bar</p>
+          </body>
+        </html>""", tmpl.generate().render())
+
+    def test_nested_include_matches(self):
+        # See ticket #157
+        dirname = tempfile.mkdtemp(suffix='genshi_test')
+        try:
+            file1 = open(os.path.join(dirname, 'tmpl1.html'), 'w')
+            try:
+                file1.write("""<html xmlns:py="http://genshi.edgewall.org/" py:strip="">
+   <div class="target">Some content.</div>
+</html>""")
+            finally:
+                file1.close()
+
+            file2 = open(os.path.join(dirname, 'tmpl2.html'), 'w')
+            try:
+                file2.write("""<html xmlns:py="http://genshi.edgewall.org/"
+    xmlns:xi="http://www.w3.org/2001/XInclude">
+  <body>
+    <h1>Some full html document that includes file1.html</h1>
+    <xi:include href="tmpl1.html" />
+  </body>
+</html>""")
+            finally:
+                file2.close()
+
+            file3 = open(os.path.join(dirname, 'tmpl3.html'), 'w')
+            try:
+                file3.write("""<html xmlns:py="http://genshi.edgewall.org/"
+    xmlns:xi="http://www.w3.org/2001/XInclude" py:strip="">
+  <div py:match="div[@class='target']" py:attrs="select('@*')">
+    Some added stuff.
+    ${select('*|text()')}
+  </div>
+  <xi:include href="tmpl2.html" />
+</html>
+""")
+            finally:
+                file3.close()
+
+            loader = TemplateLoader([dirname])
+            tmpl = loader.load('tmpl3.html')
+            self.assertEqual("""
+  <html>
+  <body>
+    <h1>Some full html document that includes file1.html</h1>
+   <div class="target">
+    Some added stuff.
+    Some content.
+  </div>
+  </body>
+</html>
+""", tmpl.generate().render())
+        finally:
+            shutil.rmtree(dirname)
+
 
 def suite():
     suite = unittest.TestSuite()
--- a/genshi/template/tests/plugin.py
+++ b/genshi/template/tests/plugin.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006 Edgewall Software
+# Copyright (C) 2006-2007 Edgewall Software
 # Copyright (C) 2006 Matthew Good
 # All rights reserved.
 #
@@ -18,7 +18,7 @@
 
 from genshi.core import Stream
 from genshi.output import DocType
-from genshi.template import MarkupTemplate, TextTemplate
+from genshi.template import MarkupTemplate, TextTemplate, NewTextTemplate
 from genshi.template.plugin import ConfigurationError, \
                                    MarkupTemplateEnginePlugin, \
                                    TextTemplateEnginePlugin
@@ -145,6 +145,23 @@
   </body>
 </html>""", output)
 
+    def test_render_fragment_with_doctype(self):
+        plugin = MarkupTemplateEnginePlugin(options={
+            'genshi.default_doctype': 'html-strict',
+        })
+        tmpl = plugin.load_template(PACKAGE + '.templates.test_no_doctype')
+        output = plugin.render({'message': 'Hello'}, template=tmpl,
+                               fragment=True)
+        self.assertEqual("""<html lang="en">
+  <head>
+    <title>Test</title>
+  </head>
+  <body>
+    <h1>Test</h1>
+    <p>Hello</p>
+  </body>
+</html>""", output)
+
     def test_helper_functions(self):
         plugin = MarkupTemplateEnginePlugin()
         tmpl = plugin.load_template(PACKAGE + '.templates.functions')
@@ -185,6 +202,15 @@
         })
         self.assertEqual('iso-8859-15', plugin.default_encoding)
 
+    def test_init_with_new_syntax(self):
+        plugin = TextTemplateEnginePlugin(options={
+            'genshi.new_text_syntax': 'yes',
+        })
+        self.assertEqual(NewTextTemplate, plugin.template_class)
+        tmpl = plugin.load_template(PACKAGE + '.templates.new_syntax')
+        output = plugin.render({'foo': True}, template=tmpl)
+        self.assertEqual('bar', output)
+
     def test_load_template_from_file(self):
         plugin = TextTemplateEnginePlugin()
         tmpl = plugin.load_template(PACKAGE + '.templates.test')
new file mode 100644
--- /dev/null
+++ b/genshi/template/tests/templates/new_syntax.txt
@@ -0,0 +1,1 @@
+{% if foo %}bar{% end %}
\ No newline at end of file
new file mode 100644
--- /dev/null
+++ b/genshi/template/tests/templates/test_no_doctype.html
@@ -0,0 +1,13 @@
+<html xmlns="http://www.w3.org/1999/xhtml"
+      lang="en">
+
+  <head>
+    <title>Test</title>
+  </head>
+
+  <body>
+    <h1>Test</h1>
+    <p>$message</p>
+  </body>
+
+</html>
--- a/genshi/template/tests/text.py
+++ b/genshi/template/tests/text.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -17,11 +17,12 @@
 import tempfile
 import unittest
 
+from genshi.template.base import TemplateSyntaxError
 from genshi.template.loader import TemplateLoader
-from genshi.template.text import TextTemplate
+from genshi.template.text import OldTextTemplate, NewTextTemplate
 
 
-class TextTemplateTestCase(unittest.TestCase):
+class OldTextTemplateTestCase(unittest.TestCase):
     """Tests for text template processing."""
 
     def setUp(self):
@@ -31,19 +32,19 @@
         shutil.rmtree(self.dirname)
 
     def test_escaping(self):
-        tmpl = TextTemplate('\\#escaped')
+        tmpl = OldTextTemplate('\\#escaped')
         self.assertEqual('#escaped', str(tmpl.generate()))
 
     def test_comment(self):
-        tmpl = TextTemplate('## a comment')
+        tmpl = OldTextTemplate('## a comment')
         self.assertEqual('', str(tmpl.generate()))
 
     def test_comment_escaping(self):
-        tmpl = TextTemplate('\\## escaped comment')
+        tmpl = OldTextTemplate('\\## escaped comment')
         self.assertEqual('## escaped comment', str(tmpl.generate()))
 
     def test_end_with_args(self):
-        tmpl = TextTemplate("""
+        tmpl = OldTextTemplate("""
         #if foo
           bar
         #end 'if foo'""")
@@ -51,16 +52,16 @@
 
     def test_latin1_encoded(self):
         text = u'$foo\xf6$bar'.encode('iso-8859-1')
-        tmpl = TextTemplate(text, encoding='iso-8859-1')
+        tmpl = OldTextTemplate(text, encoding='iso-8859-1')
         self.assertEqual(u'x\xf6y', unicode(tmpl.generate(foo='x', bar='y')))
 
     def test_unicode_input(self):
         text = u'$foo\xf6$bar'
-        tmpl = TextTemplate(text)
+        tmpl = OldTextTemplate(text)
         self.assertEqual(u'x\xf6y', unicode(tmpl.generate(foo='x', bar='y')))
 
     def test_empty_lines1(self):
-        tmpl = TextTemplate("""Your items:
+        tmpl = OldTextTemplate("""Your items:
 
         #for item in items
           * ${item}
@@ -70,10 +71,10 @@
           * 0
           * 1
           * 2
-""", tmpl.generate(items=range(3)).render('text'))
+""", tmpl.generate(items=range(3)).render())
 
     def test_empty_lines2(self):
-        tmpl = TextTemplate("""Your items:
+        tmpl = OldTextTemplate("""Your items:
 
         #for item in items
           * ${item}
@@ -87,7 +88,7 @@
 
           * 2
 
-""", tmpl.generate(items=range(3)).render('text'))
+""", tmpl.generate(items=range(3)).render())
 
     def test_include(self):
         file1 = open(os.path.join(self.dirname, 'tmpl1.txt'), 'w')
@@ -105,16 +106,159 @@
             file2.close()
 
         loader = TemplateLoader([self.dirname])
-        tmpl = loader.load('tmpl2.txt', cls=TextTemplate)
+        tmpl = loader.load('tmpl2.txt', cls=OldTextTemplate)
         self.assertEqual("""----- Included data below this line -----
 Included
             ----- Included data above this line -----""",
                          tmpl.generate().render())
-        
+
+
+class NewTextTemplateTestCase(unittest.TestCase):
+    """Tests for text template processing."""
+
+    def setUp(self):
+        self.dirname = tempfile.mkdtemp(suffix='markup_test')
+
+    def tearDown(self):
+        shutil.rmtree(self.dirname)
+
+    def test_escaping(self):
+        tmpl = NewTextTemplate('\\{% escaped %}')
+        self.assertEqual('{% escaped %}', str(tmpl.generate()))
+
+    def test_comment(self):
+        tmpl = NewTextTemplate('{# a comment #}')
+        self.assertEqual('', str(tmpl.generate()))
+
+    def test_comment_escaping(self):
+        tmpl = NewTextTemplate('\\{# escaped comment #}')
+        self.assertEqual('{# escaped comment #}', str(tmpl.generate()))
+
+    def test_end_with_args(self):
+        tmpl = NewTextTemplate("""
+{% if foo %}
+  bar
+{% end 'if foo' %}""")
+        self.assertEqual('\n', str(tmpl.generate(foo=False)))
+
+    def test_latin1_encoded(self):
+        text = u'$foo\xf6$bar'.encode('iso-8859-1')
+        tmpl = NewTextTemplate(text, encoding='iso-8859-1')
+        self.assertEqual(u'x\xf6y', unicode(tmpl.generate(foo='x', bar='y')))
+
+    def test_unicode_input(self):
+        text = u'$foo\xf6$bar'
+        tmpl = NewTextTemplate(text)
+        self.assertEqual(u'x\xf6y', unicode(tmpl.generate(foo='x', bar='y')))
+
+    def test_empty_lines1(self):
+        tmpl = NewTextTemplate("""Your items:
+
+{% for item in items %}\
+  * ${item}
+{% end %}""")
+        self.assertEqual("""Your items:
+
+  * 0
+  * 1
+  * 2
+""", tmpl.generate(items=range(3)).render())
+
+    def test_empty_lines2(self):
+        tmpl = NewTextTemplate("""Your items:
+
+{% for item in items %}\
+  * ${item}
+
+{% end %}""")
+        self.assertEqual("""Your items:
+
+  * 0
+
+  * 1
+
+  * 2
+
+""", tmpl.generate(items=range(3)).render())
+
+    def test_exec_with_trailing_space(self):
+        """
+        Verify that a code block with trailing space does not cause a syntax
+        error (see ticket #127).
+        """
+        NewTextTemplate(u"""
+          {% python
+            bar = 42
+          $}
+        """)
+
+    def test_exec_import(self):
+        tmpl = NewTextTemplate(u"""{% python from datetime import timedelta %}
+        ${timedelta(days=2)}
+        """)
+        self.assertEqual("""
+        2 days, 0:00:00
+        """, str(tmpl.generate()))
+
+    def test_exec_def(self):
+        tmpl = NewTextTemplate(u"""{% python
+        def foo():
+            return 42
+        %}
+        ${foo()}
+        """)
+        self.assertEqual(u"""
+        42
+        """, str(tmpl.generate()))
+
+    def test_include(self):
+        file1 = open(os.path.join(self.dirname, 'tmpl1.txt'), 'w')
+        try:
+            file1.write("Included")
+        finally:
+            file1.close()
+
+        file2 = open(os.path.join(self.dirname, 'tmpl2.txt'), 'w')
+        try:
+            file2.write("""----- Included data below this line -----
+{% include tmpl1.txt %}
+----- Included data above this line -----""")
+        finally:
+            file2.close()
+
+        loader = TemplateLoader([self.dirname])
+        tmpl = loader.load('tmpl2.txt', cls=NewTextTemplate)
+        self.assertEqual("""----- Included data below this line -----
+Included
+----- Included data above this line -----""", tmpl.generate().render())
+
+    def test_include_expr(self):
+         file1 = open(os.path.join(self.dirname, 'tmpl1.txt'), 'w')
+         try:
+             file1.write("Included")
+         finally:
+             file1.close()
+ 
+         file2 = open(os.path.join(self.dirname, 'tmpl2.txt'), 'w')
+         try:
+             file2.write("""----- Included data below this line -----
+    {% include ${'%s.txt' % ('tmpl1',)} %}
+    ----- Included data above this line -----""")
+         finally:
+             file2.close()
+
+         loader = TemplateLoader([self.dirname])
+         tmpl = loader.load('tmpl2.txt', cls=NewTextTemplate)
+         self.assertEqual("""----- Included data below this line -----
+    Included
+    ----- Included data above this line -----""", tmpl.generate().render())
+
+
 def suite():
     suite = unittest.TestSuite()
-    suite.addTest(doctest.DocTestSuite(TextTemplate.__module__))
-    suite.addTest(unittest.makeSuite(TextTemplateTestCase, 'test'))
+    suite.addTest(doctest.DocTestSuite(NewTextTemplate.__module__))
+    suite.addTest(unittest.makeSuite(OldTextTemplateTestCase, 'test'))
+    suite.addTest(unittest.makeSuite(NewTextTemplateTestCase, 'test'))
     return suite
 
 if __name__ == '__main__':
--- a/genshi/template/text.py
+++ b/genshi/template/text.py
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006-2007 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -11,23 +11,235 @@
 # individuals. For the exact contribution history, see the revision
 # history and logs, available at http://genshi.edgewall.org/log/.
 
-"""Plain text templating engine."""
+"""Plain text templating engine.
+
+This module implements two template language syntaxes, at least for a certain
+transitional period. `OldTextTemplate` (aliased to just `TextTemplate`) defines
+a syntax that was inspired by Cheetah/Velocity. `NewTextTemplate` on the other
+hand is inspired by the syntax of the Django template language, which has more
+explicit delimiting of directives, and is more flexible with regards to
+white space and line breaks.
+
+In a future release, `OldTextTemplate` will be phased out in favor of
+`NewTextTemplate`, as the names imply. Therefore the new syntax is strongly
+recommended for new projects, and existing projects may want to migrate to the
+new syntax to remain compatible with future Genshi releases.
+"""
 
 import re
 
-from genshi.template.base import BadDirectiveError, Template, INCLUDE, SUB
+from genshi.core import TEXT
+from genshi.template.base import BadDirectiveError, Template, \
+                                 TemplateSyntaxError, EXEC, INCLUDE, SUB
+from genshi.template.eval import Suite
 from genshi.template.directives import *
-from genshi.template.directives import Directive, _apply_directives
+from genshi.template.directives import Directive
 from genshi.template.interpolation import interpolate
 
-__all__ = ['TextTemplate']
+__all__ = ['NewTextTemplate', 'OldTextTemplate', 'TextTemplate']
 __docformat__ = 'restructuredtext en'
 
 
-class TextTemplate(Template):
-    """Implementation of a simple text-based template engine.
+class NewTextTemplate(Template):
+    r"""Implementation of a simple text-based template engine. This class will
+    replace `OldTextTemplate` in a future release.
     
-    >>> tmpl = TextTemplate('''Dear $name,
+    It uses a more explicit delimiting style for directives: instead of the old
+    style which required putting directives on separate lines that were prefixed
+    with a ``#`` sign, directives and commenbtsr are enclosed in delimiter pairs
+    (by default ``{% ... %}`` and ``{# ... #}``, respectively).
+    
+    Variable substitution uses the same interpolation syntax as for markup
+    languages: simple references are prefixed with a dollar sign, more complex
+    expression enclosed in curly braces.
+    
+    >>> tmpl = NewTextTemplate('''Dear $name,
+    ... 
+    ... {# This is a comment #}
+    ... We have the following items for you:
+    ... {% for item in items %}
+    ...  * ${'Item %d' % item}
+    ... {% end %}
+    ... ''')
+    >>> print tmpl.generate(name='Joe', items=[1, 2, 3]).render()
+    Dear Joe,
+    <BLANKLINE>
+    <BLANKLINE>
+    We have the following items for you:
+    <BLANKLINE>
+     * Item 1
+    <BLANKLINE>
+     * Item 2
+    <BLANKLINE>
+     * Item 3
+    <BLANKLINE>
+    <BLANKLINE>
+    
+    By default, no spaces or line breaks are removed. If a line break should
+    not be included in the output, prefix it with a backslash:
+    
+    >>> tmpl = NewTextTemplate('''Dear $name,
+    ... 
+    ... {# This is a comment #}\
+    ... We have the following items for you:
+    ... {% for item in items %}\
+    ...  * $item
+    ... {% end %}\
+    ... ''')
+    >>> print tmpl.generate(name='Joe', items=[1, 2, 3]).render()
+    Dear Joe,
+    <BLANKLINE>
+    We have the following items for you:
+     * 1
+     * 2
+     * 3
+    <BLANKLINE>
+    
+    Backslashes are also used to escape the start delimiter of directives and
+    comments:
+
+    >>> tmpl = NewTextTemplate('''Dear $name,
+    ... 
+    ... \{# This is a comment #}
+    ... We have the following items for you:
+    ... {% for item in items %}\
+    ...  * $item
+    ... {% end %}\
+    ... ''')
+    >>> print tmpl.generate(name='Joe', items=[1, 2, 3]).render()
+    Dear Joe,
+    <BLANKLINE>
+    {# This is a comment #}
+    We have the following items for you:
+     * 1
+     * 2
+     * 3
+    <BLANKLINE>
+    
+    :since: version 0.5
+    """
+    directives = [('def', DefDirective),
+                  ('when', WhenDirective),
+                  ('otherwise', OtherwiseDirective),
+                  ('for', ForDirective),
+                  ('if', IfDirective),
+                  ('choose', ChooseDirective),
+                  ('with', WithDirective)]
+    serializer = 'text'
+
+    _DIRECTIVE_RE = r'((?<!\\)%s\s*(\w+)\s*(.*?)\s*%s|(?<!\\)%s.*?%s)'
+    _ESCAPE_RE = r'\\\n|\\(\\)|\\(%s)|\\(%s)'
+
+    def __init__(self, source, filepath=None, filename=None, loader=None,
+                 encoding=None, lookup='strict', allow_exec=False,
+                 delims=('{%', '%}', '{#', '#}')):
+        self.delimiters = delims
+        Template.__init__(self, source, filepath=filepath, filename=filename,
+                          loader=loader, encoding=encoding, lookup=lookup)
+
+    def _get_delims(self):
+        return self._delims
+    def _set_delims(self, delims):
+        if len(delims) != 4:
+            raise ValueError('delimiers tuple must have exactly four elements')
+        self._delims = delims
+        self._directive_re = re.compile(self._DIRECTIVE_RE % tuple(
+            map(re.escape, delims)
+        ), re.DOTALL)
+        self._escape_re = re.compile(self._ESCAPE_RE % tuple(
+            map(re.escape, delims[::2])
+        ))
+    delimiters = property(_get_delims, _set_delims, """\
+    The delimiters for directives and comments. This should be a four item tuple
+    of the form ``(directive_start, directive_end, comment_start,
+    comment_end)``, where each item is a string.
+    """)
+
+    def _parse(self, source, encoding):
+        """Parse the template from text input."""
+        stream = [] # list of events of the "compiled" template
+        dirmap = {} # temporary mapping of directives to elements
+        depth = 0
+
+        source = source.read()
+        if isinstance(source, str):
+            source = source.decode(encoding or 'utf-8', 'replace')
+        offset = 0
+        lineno = 1
+
+        _escape_sub = self._escape_re.sub
+        def _escape_repl(mo):
+            groups = filter(None, mo.groups()) 
+            if not groups:
+                return ''
+            return groups[0]
+
+        for idx, mo in enumerate(self._directive_re.finditer(source)):
+            start, end = mo.span(1)
+            if start > offset:
+                text = _escape_sub(_escape_repl, source[offset:start])
+                for kind, data, pos in interpolate(text, self.filepath, lineno,
+                                                   lookup=self.lookup):
+                    stream.append((kind, data, pos))
+                lineno += len(text.splitlines())
+
+            lineno += len(source[start:end].splitlines())
+            command, value = mo.group(2, 3)
+
+            if command == 'include':
+                pos = (self.filename, lineno, 0)
+                value = list(interpolate(value, self.filepath, lineno, 0,
+                                         lookup=self.lookup))
+                if len(value) == 1 and value[0][0] is TEXT:
+                    value = value[0][1]
+                stream.append((INCLUDE, (value, None, []), pos))
+
+            elif command == 'python':
+                if not self.allow_exec:
+                    raise TemplateSyntaxError('Python code blocks not allowed',
+                                              self.filepath, lineno)
+                try:
+                    suite = Suite(value, self.filepath, lineno,
+                                  lookup=self.lookup)
+                except SyntaxError, err:
+                    raise TemplateSyntaxError(err, self.filepath,
+                                              lineno + (err.lineno or 1) - 1)
+                pos = (self.filename, lineno, 0)
+                stream.append((EXEC, suite, pos))
+
+            elif command == 'end':
+                depth -= 1
+                if depth in dirmap:
+                    directive, start_offset = dirmap.pop(depth)
+                    substream = stream[start_offset:]
+                    stream[start_offset:] = [(SUB, ([directive], substream),
+                                              (self.filepath, lineno, 0))]
+
+            elif command:
+                cls = self._dir_by_name.get(command)
+                if cls is None:
+                    raise BadDirectiveError(command)
+                directive = cls, value, None, (self.filepath, lineno, 0)
+                dirmap[depth] = (directive, len(stream))
+                depth += 1
+
+            offset = end
+
+        if offset < len(source):
+            text = _escape_sub(_escape_repl, source[offset:])
+            for kind, data, pos in interpolate(text, self.filepath, lineno,
+                                               lookup=self.lookup):
+                stream.append((kind, data, pos))
+
+        return stream
+
+
+class OldTextTemplate(Template):
+    """Legacy implementation of the old syntax text-based templates. This class
+    is provided in a transition phase for backwards compatibility. New code
+    should use the `NewTextTemplate` class and the improved syntax it provides.
+    
+    >>> tmpl = OldTextTemplate('''Dear $name,
     ... 
     ... We have the following items for you:
     ... #for item in items
@@ -36,7 +248,7 @@
     ... 
     ... All the best,
     ... Foobar''')
-    >>> print tmpl.generate(name='Joe', items=[1, 2, 3]).render('text')
+    >>> print tmpl.generate(name='Joe', items=[1, 2, 3]).render()
     Dear Joe,
     <BLANKLINE>
     We have the following items for you:
@@ -54,6 +266,7 @@
                   ('if', IfDirective),
                   ('choose', ChooseDirective),
                   ('with', WithDirective)]
+    serializer = 'text'
 
     _DIRECTIVE_RE = re.compile(r'(?:^[ \t]*(?<!\\)#(end).*\n?)|'
                                r'(?:^[ \t]*(?<!\\)#((?:\w+|#).*)\n?)',
@@ -75,8 +288,7 @@
             start, end = mo.span()
             if start > offset:
                 text = source[offset:start]
-                for kind, data, pos in interpolate(text, self.basedir,
-                                                   self.filename, lineno,
+                for kind, data, pos in interpolate(text, self.filepath, lineno,
                                                    lookup=self.lookup):
                     stream.append((kind, data, pos))
                 lineno += len(text.splitlines())
@@ -98,7 +310,7 @@
                                               (self.filepath, lineno, 0))]
             elif command == 'include':
                 pos = (self.filename, lineno, 0)
-                stream.append((INCLUDE, (value.strip(), []), pos))
+                stream.append((INCLUDE, (value.strip(), None, []), pos))
             elif command != '#':
                 cls = self._dir_by_name.get(command)
                 if cls is None:
@@ -111,9 +323,11 @@
 
         if offset < len(source):
             text = source[offset:].replace('\\#', '#')
-            for kind, data, pos in interpolate(text, self.basedir,
-                                               self.filename, lineno,
+            for kind, data, pos in interpolate(text, self.filepath, lineno,
                                                lookup=self.lookup):
                 stream.append((kind, data, pos))
 
         return stream
+
+
+TextTemplate = OldTextTemplate
--- a/genshi/tests/core.py
+++ b/genshi/tests/core.py
@@ -14,10 +14,14 @@
 import doctest
 import pickle
 from StringIO import StringIO
+try:
+    from cStringIO import StringIO as cStringIO
+except ImportError:
+    cStringIO = StringIO
 import unittest
 
 from genshi import core
-from genshi.core import Markup, Namespace, QName, escape, unescape
+from genshi.core import Markup, Attrs, Namespace, QName, escape, unescape
 from genshi.input import XML, ParseError
 
 
@@ -35,6 +39,18 @@
         xml = XML('<li>Über uns</li>')
         self.assertEqual('<li>&#220;ber uns</li>', xml.render(encoding='ascii'))
 
+    def test_render_output_stream_utf8(self):
+        xml = XML('<li>Über uns</li>')
+        strio = cStringIO()
+        self.assertEqual(None, xml.render(out=strio))
+        self.assertEqual('<li>Über uns</li>', strio.getvalue())
+
+    def test_render_output_stream_unicode(self):
+        xml = XML('<li>Über uns</li>')
+        strio = StringIO()
+        self.assertEqual(None, xml.render(encoding=None, out=strio))
+        self.assertEqual(u'<li>Über uns</li>', strio.getvalue())
+
     def test_pickle(self):
         xml = XML('<li>Foo</li>')
         buf = StringIO()
@@ -46,6 +62,10 @@
 
 class MarkupTestCase(unittest.TestCase):
 
+    def test_new_with_encoding(self):
+        markup = Markup('Döner', encoding='utf-8')
+        self.assertEquals("<Markup u'D\\xf6ner'>", repr(markup))
+
     def test_repr(self):
         markup = Markup('foo')
         self.assertEquals("<Markup u'foo'>", repr(markup))
@@ -91,6 +111,16 @@
         assert type(markup) is Markup
         self.assertEquals('<b>&amp;</b> boo', markup)
 
+    def test_mod_mapping(self):
+        markup = Markup('<b>%(foo)s</b>') % {'foo': '&'}
+        assert type(markup) is Markup
+        self.assertEquals('<b>&amp;</b>', markup)
+
+    def test_mod_noescape(self):
+        markup = Markup('<b>%(amp)s</b>') % {'amp': Markup('&amp;')}
+        assert type(markup) is Markup
+        self.assertEquals('<b>&amp;</b>', markup)
+
     def test_mul(self):
         markup = Markup('<b>foo</b>') * 2
         assert type(markup) is Markup
@@ -134,6 +164,18 @@
         self.assertEquals("<Markup u'foo'>", repr(pickle.load(buf)))
 
 
+class AttrsTestCase(unittest.TestCase):
+
+    def test_pickle(self):
+        attrs = Attrs([("attr1", "foo"), ("attr2", "bar")])
+        buf = StringIO()
+        pickle.dump(attrs, buf, 2)
+        buf.seek(0)
+        unpickled = pickle.load(buf)
+        self.assertEquals("Attrs([('attr1', 'foo'), ('attr2', 'bar')])",
+                          repr(unpickled))
+
+
 class NamespaceTestCase(unittest.TestCase):
 
     def test_pickle(self):
@@ -165,12 +207,18 @@
         self.assertEqual("QName(u'http://www.example.org/namespace}elem')",
                          repr(QName('http://www.example.org/namespace}elem')))
 
+    def test_leading_curly_brace(self):
+        qname = QName('{http://www.example.org/namespace}elem')
+        self.assertEquals('http://www.example.org/namespace', qname.namespace)
+        self.assertEquals('elem', qname.localname)
+
 
 def suite():
     suite = unittest.TestSuite()
     suite.addTest(unittest.makeSuite(StreamTestCase, 'test'))
     suite.addTest(unittest.makeSuite(MarkupTestCase, 'test'))
     suite.addTest(unittest.makeSuite(NamespaceTestCase, 'test'))
+    suite.addTest(unittest.makeSuite(AttrsTestCase, 'test'))
     suite.addTest(unittest.makeSuite(QNameTestCase, 'test'))
     suite.addTest(doctest.DocTestSuite(core))
     return suite
--- a/genshi/tests/output.py
+++ b/genshi/tests/output.py
@@ -23,6 +23,15 @@
 
 class XMLSerializerTestCase(unittest.TestCase):
 
+    def test_xml_serialiser_with_decl(self):
+        stream = Stream([(Stream.XML_DECL, ('1.0', None, -1), (None, -1, -1))])
+        output = stream.render(XMLSerializer, doctype='xhtml')
+        self.assertEqual('<?xml version="1.0"?>\n'
+                         '<!DOCTYPE html PUBLIC '
+                         '"-//W3C//DTD XHTML 1.0 Strict//EN" '
+                         '"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">\n',
+                         output)
+
     def test_doctype_in_stream(self):
         stream = Stream([(Stream.DOCTYPE, DocType.HTML_STRICT, (None, -1, -1))])
         output = stream.render(XMLSerializer)
@@ -219,7 +228,7 @@
     def test_xml_space(self):
         text = '<foo xml:space="preserve"> Do not mess  \n\n with me </foo>'
         output = XML(text).render(XHTMLSerializer)
-        self.assertEqual(text, output)
+        self.assertEqual('<foo> Do not mess  \n\n with me </foo>', output)
 
     def test_empty_script(self):
         text = """<html xmlns="http://www.w3.org/1999/xhtml">
--- a/genshi/tests/path.py
+++ b/genshi/tests/path.py
@@ -76,7 +76,7 @@
         path = Path('//*')
         self.assertEqual('<Path "descendant-or-self::node()/child::*">',
                          repr(path))
-        self.assertEqual('<elem/>', path.select(xml).render())
+        self.assertEqual('<root><elem/></root>', path.select(xml).render())
 
     def test_1step_attribute(self):
         path = Path('@foo')
--- a/genshi/util.py
+++ b/genshi/util.py
@@ -15,6 +15,11 @@
 
 import htmlentitydefs
 import re
+try:
+    set
+except NameError:
+    from sets import ImmutableSet as frozenset
+    from sets import Set as set
 
 __docformat__ = 'restructuredtext en'
 
@@ -152,7 +157,7 @@
     """
     retval = []
     for item in items:
-        if isinstance(item, (list, tuple)):
+        if isinstance(item, (frozenset, list, set, tuple)):
             retval += flatten(item)
         else:
             retval.append(item)
@@ -223,7 +228,7 @@
                     return ref
     return _STRIPENTITIES_RE.sub(_replace_entity, text)
 
-_STRIPTAGS_RE = re.compile(r'<[^>]*?>')
+_STRIPTAGS_RE = re.compile(r'(<!--.*?-->|<[^>]*>)')
 def striptags(text):
     """Return a copy of the text with any XML/HTML tags removed.
     
@@ -234,6 +239,11 @@
     >>> striptags('Foo<br />')
     'Foo'
     
+    HTML/XML comments are stripped, too:
+    
+    >>> striptags('<!-- <blub>hehe</blah> -->test')
+    'test'
+    
     :param text: the string to remove tags from
     :return: the text with tags removed
     """
--- a/setup.py
+++ b/setup.py
@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2006 Edgewall Software
+# Copyright (C) 2006-2008 Edgewall Software
 # All rights reserved.
 #
 # This software is licensed as described in the file COPYING, which
@@ -14,7 +14,7 @@
 
 from distutils.cmd import Command
 from distutils.command.build_ext import build_ext
-from distutils.errors import CCompilerError
+from distutils.errors import CCompilerError, DistutilsPlatformError
 import doctest
 from glob import glob
 import os
@@ -25,102 +25,33 @@
     Feature = None
 import sys
 
-
-class build_doc(Command):
-    description = 'Builds the documentation'
-    user_options = [
-        ('force', None,
-         "force regeneration even if no reStructuredText files have changed"),
-        ('without-apidocs', None,
-         "whether to skip the generation of API documentaton"),
-    ]
-    boolean_options = ['force', 'without-apidocs']
-
-    def initialize_options(self):
-        self.force = False
-        self.without_apidocs = False
-
-    def finalize_options(self):
-        pass
-
-    def run(self):
-        from docutils.core import publish_cmdline
-        from docutils.nodes import raw
-        from docutils.parsers import rst
-
-        docutils_conf = os.path.join('doc', 'conf', 'docutils.ini')
-        epydoc_conf = os.path.join('doc', 'conf', 'epydoc.ini')
-
-        try:
-            from pygments import highlight
-            from pygments.lexers import get_lexer_by_name
-            from pygments.formatters import HtmlFormatter
-
-            def code_block(name, arguments, options, content, lineno,
-                           content_offset, block_text, state, state_machine):
-                lexer = get_lexer_by_name(arguments[0])
-                html = highlight('\n'.join(content), lexer, HtmlFormatter())
-                return [raw('', html, format='html')]
-            code_block.arguments = (1, 0, 0)
-            code_block.options = {'language' : rst.directives.unchanged}
-            code_block.content = 1
-            rst.directives.register_directive('code-block', code_block)
-        except ImportError:
-            print 'Pygments not installed, syntax highlighting disabled'
-
-        for source in glob('doc/*.txt'):
-            dest = os.path.splitext(source)[0] + '.html'
-            if self.force or not os.path.exists(dest) or \
-                    os.path.getmtime(dest) < os.path.getmtime(source):
-                print 'building documentation file %s' % dest
-                publish_cmdline(writer_name='html',
-                                argv=['--config=%s' % docutils_conf, source,
-                                      dest])
-
-        if not self.without_apidocs:
-            try:
-                from epydoc import cli
-                old_argv = sys.argv[1:]
-                sys.argv[1:] = [
-                    '--config=%s' % epydoc_conf,
-                    '--no-private', # epydoc bug, not read from config
-                    '--simple-term',
-                    '--verbose'
-                ]
-                cli.cli()
-                sys.argv[1:] = old_argv
-
-            except ImportError:
-                print 'epydoc not installed, skipping API documentation.'
-
-
-class test_doc(Command):
-    description = 'Tests the code examples in the documentation'
-    user_options = []
-
-    def initialize_options(self):
-        pass
-
-    def finalize_options(self):
-        pass
-
-    def run(self):
-        for filename in glob('doc/*.txt'):
-            print 'testing documentation file %s' % filename
-            doctest.testfile(filename, False, optionflags=doctest.ELLIPSIS)
+sys.path.append(os.path.join('doc', 'common'))
+try:
+    from doctools import build_doc, test_doc
+except ImportError:
+    build_doc = test_doc = None
 
 
 class optional_build_ext(build_ext):
     # This class allows C extension building to fail.
+    def run(self):
+        try:
+            build_ext.run(self)
+        except DistutilsPlatformError:
+            self._unavailable()
+
     def build_extension(self, ext):
         try:
             build_ext.build_extension(self, ext)
         except CCompilerError, x:
-            print '*' * 70
-            print """WARNING:
+            self._unavailable()
+
+    def _unavailable(self):
+        print '*' * 70
+        print """WARNING:
 An optional C extension could not be compiled, speedups will not be
 available."""
-            print '*' * 70
+        print '*' * 70
 
 
 if Feature:
@@ -137,7 +68,7 @@
 setup(
     name = 'Genshi',
     version = '0.5',
-    description = 'A toolkit for stream-based generation of output for the web',
+    description = 'A toolkit for generation of output for the web',
     long_description = \
 """Genshi is a Python library that provides an integrated set of
 components for parsing, generating, and processing HTML, XML or
Copyright (C) 2012-2017 Edgewall Software