evoque – managed eval-based freeform templating
Evoque is a full-featured generic text templating system for python using
a simple $-substitution syntax and providing support for flow control, nested
templates, overlays, inter-template addressing and invocation, cache management,
arbitrary python expressions, all python % operator formatting,
restricted execution, automatic cross-site scripting protection, advanced
encoding guessing algorithm, and more.
Allowing only python expressions and a managed evaluation namespace, Evoque
offers a surprising level of simplicity, versatility and performance.
Buzz
- Full-featured pure python templating engine / 992 SLOC
- Python 2.4, 2.5, 2.6 and 3.0
- Unicode
- Simplicity
- Restrictable execution / sandbox
- Data-driven template inheritance / runtime base template selection
- Open-ended source or target file formats
- Templates are managed via Collections in a Domain / cascading defaults
- Domain-wide template names / addressing
- Automatic input quoting / XSS protection
- Speed
Snapshot
$overlay{name=site_base} $begin{table_row} $for{ col in row } <td>${col}</td>\ $else <td class="empty" colspan="7">empty row</td> $rof $end{table_row} <table> $for{ i, row in enumerate(rows) } <tr$if{i%2} class="odd"$fi> #[ "odd" rows get a special style ]# $evoque{#table_row} </tr> $rof </table> $evoque{disclaimer, collection="legals"} $test{% site_base="site.html", rows=[("a", "b", 3.0, {"one":1}, "<escape-me/>", "i", "j")] %}
Usage from within a Python application
Templates are always part of a Collection in a Domain —
a Domain and a default Collection instance are always created,
be it explicitly or implicitly.
The preferred way to load/retrieve a Template is via a Domain instance;
the following code implicitly creates the default Collection instance:
domain = Domain("/home/user/templates") t = domain.get_template("snap.html") print t.evoque(vars())
It may sometimes be more convenient to instantiate directly;
the following code implicitly creates the Domain and the default
Collection instances:
t = Template("/home/user/templates", "snap.html") print t.evoque(vars())
Basic benchmark
A basic benchmark, consisting of a small template
that does a variety of standard templating tasks such as looping,
calling nested and external sub-templates, escaping of data, etc.
For each run, time is averaged over 2000 renderings, and the
best time of 4 runs is retained.
All times are in ms.
What you should want is automatic quoting and less time.
quoting | qpy 1.7 | evoque 0.4 | mako 0.2.4 | genshi 0.5.1 | |
---|---|---|---|---|---|
automatic | 0.104 | 0.339 | 0.421 | 2.706 (c) | |
automatic | R(a) | 0.386 | |||
manual | 0.318 (b) | 0.387 | |||
none | 0.262 | 0.308 |
(a) Restricted —
qpy 1.7 /
mako 0.2.4 /
genshi 0.5.1
offer no support for restricted execution or sandboxing
(b) manual quoting implies no qpy
(c) xml mode
Feature highlights
Evoque is generic, meaning it can be made to output
text in any format and run in any context
(stand-alone or integrated within an application framework)
with state-of-the-art features such as:
unicode, dynamic overlays, template caching,
format-extensible automatic quoting,
in-process sandbox —
while still remaining small, simple and extremely fast.
Small footprint |
Coming in at 992 SLOC. |
---|---|
Unicode |
All internal processing is done on unicode string instances.
Evoque includes an |
Template addressing / collections in a domain |
Templates are organized in collections in a domain, |
Mixing/nesting directives |
Conditionals, loops, nested templates, |
Automatic X/HTML input quoting / XSS protection |
Thanks to
To benefit from this feature Qpy needs to be installed — |
Quoted-No-More / Extensible automatic quoting |
The unique and powerful automatic-input-quoting-once-and-only-once |
Expressions only |
No python statements, nothing is ever exec’ed, |
Restricted execution |
Evaluation is within a managed, and |
Managed evaluation namespace |
Applications may |
All python % operator formatting |
Expressions have the form ${expr !format} where the “!format” |
Good error handling |
Good |
Template caching |
Templates, once compiled and loaded, are cached in memory. |
Data-driven template inheritance |
See how easy it is to dynamically select |
Open-ended source or target file formats |
Template source files as well as generated output may be in any text format. |
Introspectable templates |
Easily introspectable templates, of their signatures and evaluation |
Whitespace control |
A few powerful options to enable you to effectively format the template |
Comments |
Evoque supports single-line, multi-line, and even nested comments — In addition, python expressions may of course |
Simplicity |
A trivially memorable syntax, |
Speed |
The simplicity of the implementation, and the awesome python builtins and |
beauty in the details, evoque’s unique features
For the exquisite pleasure of all of you who appreciate the many
little details that make up a beautiful design,
here are just some of unique qualities of Evoque
that bubble up to user visibility as a feature
in some big or small way.
Note that this is a sampler only of features
that are exclusive and unique to Evoque i.e.
not any of the big and important Evoque features —
that probably every other templating engine should also have
[a]
even if surprisingly few actually do e.g.
sandbox, complete unicode, automatic escaping.
In some cases, contrasting a feature with how other
templating system might support it
helps to better appreciate the feature or its intention —
for these cases, we pay honour to the extremely capable
Mako 0.2.4
that amongst text templating systems is probably the
most comparable to Evoque.
small yet ultra-capable |
One indication of size is the SLOC count
Another indication of size is the number of
Beyond the template constructs, that same small number of SLOC
Small is of course naturally accompanied by simple, |
---|---|
automatic once-only escaping for any output format |
Evoque is constructed on the quoted-no-more |
clean template inheritance |
An Evoque template needs no modifications to be used as a base template |
positive and negative template space |
Unique to Evoque is the concept of |
line-agnostic control structures |
All controls structures and directives may be |
one syntax |
Evoque logic in templates is specified with directives and expressions. |
conditional decorators |
An open-ended mechanism to conditionally intercept input and/or |
template auto tests |
A template may declare multiple |
all python % operator expression formatting |
Expressions may take advantage of |
nested comments |
Surprisingly rare! |
nicer to the file-system |
Evoque’s template caching |
not inappropriately pythonic |
Some template engines pride themselves with being pythonic, |
no implementation clutter |
Usage of evoque is not cluttered by underlying implementation details |
Python 3.0 |
Since version 0.4 |
[a] Roland Koebler identifies such
a list of bread and butter features
that every templating engine should have.
[b] Source Lines of Code (SLOC) are counted with
SLOCCount.
Template syntax
definition: An evoque template is either
(a) a textual file or
(b) a string in memory or
(c) a begin/end delineated block in a textual file or a string,
each optionally containing additional evoque-specific markup.
Entire template syntax at a glance
$$ -> escape, replaced with $
${expr} -> substitution, like %(expr)s
${expr!format} -> substitution, like %(expr)format
directives
$if{expr} … $elif{expr} … $else … $fi
$for{item in items} … $else … $rof
$begin{label} … $end{label}
$prefer{raw=False, data=None, quoting=None, filters=None}
$test{**kw}
$evoque{name, src=None, collection=None, raw=False,
quoting=None, input_encoding=None, filters=None, **kw}
$overlay{name, src=None, collection=None, space=”positive”}
comments
#[ template comment: single-line, multi-line, nested ]#
${ expr # python single-line comment }
delimeters
either “{” and “}” or “{%” and “%}”, liberally interchangeable
whitespace
“\” at end-of-line consumes the following newline
built-in callables
${evoque(name, **kw)}
${inspect()}
Explanations
Expressions
Expressions have the form ${expr !format} where the “!format”
is optional, and defaults to “!s”, i.e. default string rendering. The possible
values for format are whatever is allowed by the
python’s
string formating operations.
The expr itself may be any valid python expression, that will be evaluated
within the passed context (that may optionally be restricted). For example, the
trivial template:
<p>${amount} ${amount!.2f}</p>
will produce the following output when rendered with
amount=1.0/3
:
<p>0.333333333333 0.33</p>
Directives
Directives have the form $directive{expr}.
Syntactically they are same as an expression but have an additional
qualifying keyword
inserted between the $
and the opening {
.
See dedicated discussion on directives in general and for each
directive in the directives section.
Literals: “$”, “{“, “}”, “!”, “\”
A literal “$” must always be escaped.
A single “$” always signals the beginning of a substitution or a directive,
e.g $expr
has no specific meaning for Evoque and will
give a SyntaxError
(the “$” used to delineate any expanded svn keywords are exempted from this).
Should always use either $$expr
or ${expr}
instead,
depending on what is desired.
A literal “\” at end of a line slurps the newline character but
a literal “\” followed by white space (only) until the first newline
character is put out as is but without the white space, i.e.
"\ \n"
outputs
"\\n"
.
All other literals within the text are never escaped.
Within an expression it is sometimes necessary to use literal braces
e.g. to specify a dict value in place, such as key={"a":1}
,
or a literal piece of CSS such as "tag.class{display:inline}"
.
The alternative “{%” and “%}” expression delimeters are provided
to comfortably allow for this.
Whitespace
A “\” at end-of-line consumes the following newline, providing a basic
whitespace control. In addition to this, the
slurpy_directives=True
on domain
or on
each collection
will
consume all (non-newline) leading and trailing whitespace
on the same line plus — only on the left side — the initial newline.
In addition, all leading or trailing whitespace
within expressions, i.e. between the curly braces,
even if multi-line, is always ignored.
Comments
Single or multi line comments are delineated with
#[
and ]#
,
respectively. For the convenience of easily commenting out
a template section (even if that section itself already
contains comments) comments may be nested. This however
necessitates a minimum syntax to be respected, namely that
comment openings and closing are balanced.
Built-in callables
Any callable set on the evaluation context, either with
domain.set_on_globals()
or by setting it on a
locals
dict that is then passed as a parameter to
template.evoque(locals)
,
may be called as an expression, e.g.
${my_callable(params)}
.
Evoque provides the following built-in callables.
inspect(output=True)
Pretty prints an overview of the evaluation namespace,
namely the context’s globals/locals and the template’s
expressions needing evaluation.
When output is set to False, the output is written
to domain.log.info()
instead of rendered as
part of the template output.
The output of inspect()
is of course escaped as per the
quoting setting of the template. However, templates should normally also
wrap this call into an appropriate markup container
e.g. wrap in a <div class="code">
element
if calling from an HTML context.
evoque(name, **kw)
This is perfectly equivalent to the directive form of
$evoque{name, **kw}
.
The difference is that in the callable variation
it is possible to intercept the output and
process it further via any other callable —
but note that it is recommended that such
processing be specified via the filters
template parameter.
Directives
Directives have the form $directive{expr}.
What expr may be depends on the directive e.g.
for $if{expr}
the expr
is a normal expression evaluated as a bool
while for
$for{expr in sequence}
it must have this specific form.
A directive clause does not always have an expr,
and in these cases the curly braces are not present,
e.g. $else
, $fi
, $rof
.
Directives must respect their specific context,
e.g. if a $for{}
is opened inside a nested template,
then it must be closed with $rof
within the
same nested template.
Directives, in general, may be liberally intermixed and nested.
A begin/end
block may include other begin/end
blocks, if
blocks, for
blocks,
an overlay
declaration, etc.
However, specific constraints apply,
e.g. only one prefer
and overlay
directives are allowed per template, nested or not, and they must be
declared at top level of the template to which they pertain.
There is no significance to whether directives and any
clauses are specified on a single line or on separate lines.
This has benefits such as allowing an inlined if/else
e.g. to calculate the value of an HTML attribute,
just as easily as multi-line if/else
blocks
to control which section of a template gets rendered.
conditional if
The conditional if/elif/else
construct,
apart from the closing $fi
,
behaves just like you would expect i.e. as in python,
with optional (multiple) $elif{}
‘s and an optional
concluding $else
.
The condition expression may be any python expression, that
must evaluate without errors in the supplied namespace.
As for all directives, being on a single line or spanning
many lines has no relevance.
for loops
Again, the for
loop directive behaves like you would
expect (like in python) except for additional $else
option for the case of an empty sequence
(note the difference to python’s $else
that is executed at end unless loop exited with
break
statement).
One should note that the statement of the loop must be made up
of an iterable on the right side of an in
keyword,
and whatever is on the left side of the in
statement
is the loop variable or variables that get set on the evaluation
context on each iteration. This may be a single atomic variable,
a flat sequence of variables, or even something more complex.
Here’s an example of a 2-level tuple as left part of a
for loop statement:
$prefer{% data={"items":[("Apples", 7), (789, 9), ("Blue", 17)]} %} $if{items} <ul> $for{i, (item, code) in enumerate(items)} <li$if{i+1==len(items)} class="last"$fi>${code}:${item}</li> $rof </ul> $fi
That when rendered will produce:
[KeyError: ‘auto’: File “/home/admin/py-packages/evoque/domain.py”, line 108, in get_collection
return self.collections[name]
: EvalError(evoque(name=u’for_seq_items.txt’, collection=”auto”,
raw=True, filters=[evoque_html_highlight]))]
begin/end blocks: sub-templates
Nested templates are the def, the macro,
the callable block, the sub-template of Evoque.
The $begin{label}
and the
$end{label}
pair of directives
define a sub-template, that is a template nested within a
template (file or string).
The label
may be considered
just as a named anchor, a destination inside the
file to which one can point to directly–a sub-template is
addressed via the label, using a syntax
similar to fragment identifiers of a URL, i.e. #label
.
When being addressed locally, i.e. from within the body of the
same parent (sub-)template,
a sub-template may be addressed with just #label
.
Outer sub-templates (i.e. sub-templates at the top level)
are addressable from any other template;
Inner sub-templates (i.e. sub-templates within
sub-templates) are private to their direct parent sub-template
i.e. their scope is local to their parent sub-template.
Parameters
There is really only one parameter, and that is the
label that is what is used to address the template.
…template body…
$end{ label }
Execution
To be executed, they are evoque’d:
This will evaluate the nested template using a copy
of the caller’s context.
Note that a nested template is a full-fledged template in its own right,
and may therefore specify its own $prefer{}
and $overlay{}
directives i.e. it may specify its own preferences e.g. its own default data
to use should the caller’s context not specify a needed variable,
or whether it is itself an overlay over another template.
Nested template principles
- A label (for a
begin/end
block) that defines
a nested template is unique within the parent template’s scope. - All
begin/end
blocks contained in a template must
be properly nested such that they do not overlap (same
well-formedness principle as in XML). -
Inner sub-templates may only be addessed locally
i.e via#label
from within the code body of the
sub-template in which they are defined.
Note, in case of the parent sub-template being an overly, and
it does not locally define a#label
sub, the inner sub-template
is of course looked up from the overlaid template. -
Outer sub-templates may be addessed both locally,
as for inner sub-templates, as well as externally from within
other templates–in which case the information to identify the top-level
template must also be specified
i.ename#label
for an explicitly named template,
or by something liketemplate.html#label
for a template
implicitly named by its file name. - Within overlay templates
it is possible to explicitly specify what level the lookup for a
locally addressed nested template should start from
i.e. if the name starts with a single “#” then
start lookup from current level,
if with “##” then from one level down,
if with “###” then from two levels down, etc. - If only one of the
begin
orend
directives is present, then the other is implied by, respectively,
the beginning or the end of the string. -
An outer nested template has all the
characteristics of a standalone file or string template.
The only apparent difference is in how it is
addressed–as sub-templates are addressed with#label
,
one can consider that a standalone template is the special-case
sub-template having the empty string as the label
(and with both thebegin
andend
directives being omitted and thus being simply just implied
by the beginning and end of the template string). - A nested template is not automatically rendered when the template
that defines it is rendered; it must be explicitly evoque’d. - Everything already established about directives applies, e.g.
bothbegin
andend
may all be on same line,
or each may span many lines;
any of the two allowed delimeter pairs may be used;
any whitespace around the label is ignored, etc.
evoque: rendering a template
Any Evoque template, be it file-based or
string-based, top-level or nested, is addressable from any
other Evoque template in the domain.
From within a template, $evoque{name, **kw}
is the directive that gets and renders a template —
it combines the full functionality of the two methods you would
need to call to do the same thing from within an application, i.e. the
methods
Collection.get_template()
and
Template.evoque()
.
Parameters
quoting, input_encoding, filters, **kw }
name: required, the name of the template to evoque()
src, collection, raw, quoting, input_encoding, filters
: behave as in Domain.get_template()
**kw : any additional keyword args are set on evaluator’s locals
Rules
The only required parameter is name, that must always be
specified first, and most often the call is as simple as:
$evoque{name}
.
Note that for convenience the name
parameter may be
specified as an unquoted string literal i.e. when the name=
qualifier is not explicitly stated, the string found will be interpreted as
a literal even if it is not enclosed in quotes.
For example, for a template with name my_template
, using any of
the following 3 forms is perfectly equivalent:
$evoque{“my_template”}
$evoque{name=”my_template”}
But:
will cause the runtime evaluation
of a my_template
variable that will be used as the name
.
The evoque
directive gets the named template from the
specified collection — if collection=None
, then the
calling template’s own collection is used, otherwise
collection
must name an already existing collection.
Template may still need to be loaded.
If it is already loaded, the template is retrieved from the
cache, subject to cache staleness checking as per the
collections’s cache settings.
The src, input_encoding
parameters
are only used in the event that the template needs to be loaded:
- if template is not yet loaded then it is assumed to be a file-based template
and the location of the file to load is given by joining the collection’s
base path and the value ofsrc
. - if
src=None
then name is interpreted as the
location of file to load.
All parameters other than name, src, collection
,
if not specified, will be set to the collection’s default.
The raw
parameter determines whether the raw template
string is to be returned, without any evaluation, e.g. for the purpose of
editing the template string itself. Note that if the template is not yet
loaded and raw=True
, then it will be loaded but not compiled.
See the raw source howto for more details.
The quoting
parameter determines what the default
quoted-no-more quoting class for the template is
(if the template is not yet loaded) and/or which
quoting class to use for this evoque’ation.
Any specified filters
,
that must be either None or a sequence of filter functions,
are executed after evaluation.
Filters must of course be available in the evaluation space.
Built-in callable
The $evoque{name, **kw}
directive is also
provided as the perfectly equivalent built-in callable:
-
“name” is now a normal python string so may be evaluated,
i.e. if literal, then it must be quoted. - may be intermingled with other callables, python builtins,
custom filters, etc. For example:
${markdown(evoque("my_mardown_template"))}
overlay
The $overlay{}
directive defines a dependency chain of templates,
super-imposing a template on top of another.
This functionality is often called template inheritance —
a term that may however be a little misleading given the very
specific connotations from object-oriented programming.
Nonetheless, a great feature of evoque overlays is that they are in many
ways similar in behaviour to the inheritance paradigm of object-oriented
programming, as for example the nice feature that an evoque template does
not care or even know about whether it is being used as a base template
to an overlay.
Layered positive and negative spaces
A simple way to think of an overlay is as a graphic, with layers that may be
transparent, and having positive and negative space. The positive space are
the sub-template objects on the page, i.e. the top-level
begin/end
blocks defined by the template.
The negative space is everything else, i.e. all markup that is not
contained in any top-level begin/end
block.
An overlay template may declare that it overlays the
space="positive"
or the space="negative"
of the overlaid template:
- A positive-space overlay is one that may override any objects in
the positive-space, i.e. top-level sub-template objects.
Any markup in the negative-space of such a template is ignored.
By default, overlays are positive-space. -
A negative-space overlay is required to supply the
negative-space for the template. That negative space may evoque
sub-templates from the overlay chain below as if they were
defined locally. Any sub-template defined below that is not
evoque’d from the supplied negative-space will not be rendered.
Naturally, such a template may of course also override any
top-level sub-templates that make up its positive space.
Examples
The pages below offer some overlay scenario examples.
These examples are live, in the sense that the
templates and generated output are included automatically
from the Evoque unit tests,
with zero touch-up or modification.
Overlay directive parameters
The first 3 parameters are the same as for the
evoque directive
plus an additional space
keyword
that as we have seen may be
either positive
or negative
:
name, src, collection
: behave in same way as for $evoque{} directive
space : “positive” : either(“positive”, “negative”)
Additional notes about overlays
- An overlay directive declares that this template overlays
the specified template, in either a positive-space or in a negative-space
mode. - Only one overlay directive per template is allowed.
- Any template may be overlaid, and overlaid templates are just normal
templates requiring no modifications. - An overlaid template may itself overlay another template.
- In an overlay, overlaid templates may be addressed with multiple
leading#
characters, to indicate deeper levels into the
overlay chain, i.e.:#label
tells to start lookup from self##label
tells to start lookup from one level below###label
tells to start lookup from two levels below, etc.specified_name#label
continues to work as in any other template, of
course, irrespective of whether the template given byspecified_name
participates in the overlay chain or not.
overlay example – basic
Here is a simple example, showing a frequently occuring case
of having a template overlay another’s positive-space.
We use a base template that defines a page layout, and three sub-templates.
We overlay the base.html
template and override one of
the three sub-templates, namely #content
.
The templates
base.html
<html> <head><title>template = ${title}</title></head> <body> <span> -ve space (base.html) </span> $begin{header} +ve space: base header $end{header} $begin{content} +ve space: base content $end{content} $begin{footer} +ve space: base footer $end{footer} <table class="layout"> <tr><td>$evoque{#header}</td></tr> <tr><td>$evoque{#content}</td></tr> <tr><td>$evoque{#footer}</td></tr> </table> </body> </html>
- just a normal template, perfectly usable as is
- defines 3 nested templates, header, content, footer, and evoques them
- no need to be aware of any overlays over it
overlay.html
$overlay{base.html} -ve space (overlay.html) $begin{content} overlay ${parametrized} content $end{content} -ve space (overlay.html)
- positive overlay template on base.html, own negative space is ignored
- overrides #content, one of the 3 nested templates defined by base.html
- if we add definitons for other nested (except #header and #footer
that are defined in base) they would be ignored - when evoque’d directly, this template is rendered with:
- the -ve space from base.html
- own #content
- the #header and #footer from base.html
Output
Rendering the overlay:
name = "overlay.html" domain.get_template(name).evoque(title=name, parametrized="HAPPY")
Will give the following output:
[KeyError: ‘auto’: File “/home/admin/py-packages/evoque/domain.py”, line 108, in get_collection
return self.collections[name]
: EvalError(evoque(name=u’overlay.txt’, collection=”auto”, raw=True, filters=[evoque_html_highlight]))]
overlay example – positive
Let’s extend the simple example to add an intermediate template, so we
therefore have an overlay chain consisting of 3 templates, namely
overlay_chain_pos.html
over overlay_mid.html
over the unmodified base.html
(from simple example).
The templates
overlay_mid.html
$overlay{base.html} -ve space (overlay_mid.html) $begin{footer}<span>overlay_mid footer</span>$end{footer} -ve space (overlay_mid.html)
- an intermediate positive overlay on base.html
- the -ve space from base.html is imposed
- overrides one +ve block, #footer
overlay_chain_pos.html
$overlay{overlay_mid.html} <html> <head><title>template = ${title}</title></head> <body> <span> -ve space (overlay_chain_pos.html) </span> $begin{content} overlay_chain_pos ${parametrized} content $end{content} <div class="header">$evoque{#header}</div> <div class="content">$evoque{#content}</div> <div class="footer">$evoque{#footer}</div> </body> </html>
- overrides the positive space of overlay_mid.html, redefining #content
- when evoque’d directly, this template is rendered with:
- the -ve space from below
- own #content
- the #header and #footer from below
Output
Rendering the overlay:
name = "overlay_chain_pos.html" domain.get_template(name).evoque(title=name, parametrized="HAPPY")
Will give the following output (note negative space used is that from base.html):
[KeyError: ‘auto’: File “/home/admin/py-packages/evoque/domain.py”, line 108, in get_collection
return self.collections[name]
: EvalError(evoque(name=u’overlay_chain_pos.txt’, collection=”auto”, raw=True, filters=[evoque_html_highlight]))]
overlay example – negative
Let’s modify the positive chain example a little to show a negative overlay.
We add the overlay_chain_neg.html
template that is a negative
overlay on the unmodified overlay_mid.html
+ base.html
base templates (from previous example). This example also shows how to specify
specific levels in the overlay chain at which to start lookup.
overlay_chain_neg.html
$overlay{overlay_mid.html, space="negative"} <html> <head><title>template = ${title}</title></head> <body> <span> -ve space (overlay_chain_neg.html) </span> $begin{content} overlay_chain_neg ${parametrized} content $end{content} $begin{footer}<span>overlay_chain_neg footer</span>$end{footer} <div class="header">$evoque{#header}</div> <div class="content">$evoque{#content}</div> <div class="refootered"> <div class="own">$evoque{#footer}</div> <div class="overlaid">$evoque{##footer}</div> <div class="named">$evoque{base.html#footer}</div> </div> </body> </html>
- a negative overlay, thus using own negative space
- redefines #content and #footer
- evoques all 3 #footer templates in the overlay chain, showing how to:
- start lookup from self
- start lookup from “one level below”
- explicitly evoque a named template
Output
Rendering the overlay:
name = "overlay_chain_neg.html" domain.get_template(name).evoque(title=name, parametrized="HAPPY")
Will give the following output:
[KeyError: ‘auto’: File “/home/admin/py-packages/evoque/domain.py”, line 108, in get_collection
return self.collections[name]
: EvalError(evoque(name=u’overlay_chain_neg.txt’, collection=”auto”, raw=True, filters=[evoque_html_highlight]))]
autoload preferences
The $prefer{}
directive is a
declaration of autoload preferences, from within a template.
Only one per template is allowed.
When template is fetched on the fly e.g. when evoque’d
from another template, it may either have already been
loaded, compiled, and cached or it may still need to be
looked up and compiled — in this latter case all parameters
that may affect how the template is compiled are needed.
Precisely for such a case, the $prefer{}
directive
provides the convenience of specifying default parameter values
that will be used if corresponding values are not specified
when the template is loaded.
Parameters
The parameters are a subset of those of the
evoque directive.
A parameter of the prefer directive is only taken into
consideration if:
- it has not been explicitly set when the template was first loaded
i.e. as a parameter of the
get_template()
or$evoque{}
call that
actually loads the template —
as opposed to subsequent calls that only fetch a template from the cache; - the preferred value is different than the cascaded
default value from the collection.
raw : may trigger template recompile
data : may set value of template.data
quoting : may trigger template recompile
filters : may set value of template.filters
For details of each of these parameters, see the
evoque
directive page.
auto testing a template
The $test{}
directive specifies test data
within the template itself, that will be used to evaluate
the template when template.test()
is called.
test
directives have no effect on output generated
when a template is evoque’d.
Parameters
May specify sample data, as keyword parameters that the template
may need.
Positional arguments are not supported — only keyword args.
From an application, any tests are executed with:
Example
$if{ is_good } <p class="yes">Good for you, ${first}!</p> $elif{ is_adult } <p class="adult">Think again, ${last}.</p> $else <p class="minor"> That is not quite right ${first}. Try again! </p> $fi $test{is_good=True, is_adult=True, first="Jo", last="Dough"} $test{is_good=False} $test{is_adult=False}
Executing template.test()
for the above template will thus
evaluate it 3 times, one for each test directive specified. Any parameter
not specified in a test
directive will inherit its value from
the previous test
directive.
In this case the test data provides for coverage of all the possible
logical paths, so every piece of the template is exercised.
Combined with template’s default data
Templates may be set-up to define and remember own default data.
This may be done either at initialization (data init parameter)
or via a $prefer{data=dict(**kw)}
directive.
If a template’s data attribute is set, the test()
method will use that value as the starting point for the test data cascade.
To illustrate this relation, here’s the example above, adjusted with a
$prefer{}
directive and still giving identical
test()
results:
$prefer{ data=dict(first="Jo", last="Dough") } $if{ is_good } <p class="yes">Good for you, ${first}!</p> $elif{ is_adult } <p class="adult">Think again, ${last}.</p> $else <p class="minor"> That is not quite right ${first}. Try again! </p> $fi $test{is_good=True, is_adult=True } $test{is_good=False} $test{is_adult=False}
Test directive principles
- Specify sample test data for the template, nested or not.
- May specify as many as desired, execution will cascade through them in
the specified order, i.e. if a kw arg is not specified in current test,
then its last defined value will be used for this test. - If template defines any default
data
, then that data is
used as initial data for the tests
(see Combined with template’s default data, below). - Executed simply with
template.test()
, and if any
evaluation errors occur they are always raised irrespective
of the setting fordomain.errors
. -
template.test()
returns a list of responses,
one for eachtest
directive. If notest
directive is specified, a test will always be attempted with either
the template’s default data or with no data. -
template.test()
on araw
template
returnsNone
. - A
test
directive must occur at the template’s top level. - May also be used to prime the
template.evaluator.codes
cache.
Usage from within a Python application
Templates are always part of a Collection in a Domain —
a Domain and a default Collection instance are always created,
be it explicitly or implicitly.
The preferred way to load/retrieve a Template is via a Domain instance;
the following code implicitly creates the default Collection instance:
domain = Domain("/home/user/templates") t = domain.get_template("snap.html") print t.evoque(vars())
It may sometimes be more convenient to instantiate directly;
the following code implicitly creates the Domain and the default
Collection instances:
t = Template("/home/user/templates", "snap.html") print t.evoque(vars())
Note however that implicitly creating the Domain and the default
Collection in this way will not allow you to change the default
settings for the
restricted
,
errors
,
log
,
cache_size
,
auto_reload
,
slurpy_directives
Domain and Collection attributes.
Evoque is conceived with safety and security in mind.
Under no circumstances, for example, is it ever possible to evoque a
template from a file that is not within a declared collection.
Given that there is no special distinction between templates
and content, template collection root directories are treated
in the same way that a web server treats its document-root.
The template’s collection and domain
Given a template
instance, you can always get the associated
collection
and domain
with:
template.collection template.collection.domain
To create the (default) collection first, thus having an opportunity to
name it explicitly, you do:
from evoque.collection import Collection collection = Collection(None, "default_collection", "/home/user/templates") domain = collection.domain assert collection is domain.get_collection("default_collection") assert collection is domain.get_collection()
Additional useful design principles
- Templates are organized in Collections that belong to a Domain.
- Collections have a root folder — all text files below are template-addressable.
- All collections are explicitly named — decouples template addressing from
deployment paths and protects sensitive path info within templates. - The creation of the Domain also creates the default Collection, that is named
“” (the empty string). The default collection may be given
another name, by simply creating it first. - Every template instance is associated to one and only one collection.
- All templates are assigned a nick name, unique within the
collection: memory-based templates are always named explicitly;
for file-based templates the default name is their
collection-root-relative or c-rel locator. - The src or location of any collection-qualified template
is always specified as c-rel. - A previously loaded template may be retrieved by its name and the
collection. - The eval globals dict belongs to Evoque and/or
the Client Application, and is shared by all templates
in a domain — it is not affected by template
evaluations i.e. it never changes as a side-effect
of a template being rendered. - The eval locals dict belongs to the document
being rendered, and is passed down the nested execution scope,
possibly cloned and modified each time.
domain, collection, template
A template is always part of a collection
in a domain.
The implications of this are that those related objects are always
created, even if we are still free to create them
explicitly in any order we prefer.
Another implication is consistency and simplicity of the
implementation, reducing the need to handle other scenarios.
The preferred usage is to first explicitly create the Domain instance,
that will by default create the default empty string named
Collection.
Domain
class Domain(object): """ A registry of collections """ def __init__(self, default_dir, restricted=False, errors=3, log=get_log(), # defaults for Collections cache_size=0, auto_reload=60, slurpy_directives=True, # defaults for Collections (and Templates) quoting="xml", input_encoding="utf_8", filters=[]): """ default_dir: either(str, Collection) abs path, or actual default collection instance restricted: bool restricted evaluation namespace errors: int ["silent", "zero", "name", "render", "raise"] log: the logging.getLogger("evoque") logger; may be pre-initialized and passed in, or adjusted as needed after initialization. Default settings (via loggin.basicConfig()) are: handler=StreamHandler() level=logging.INFO format="%(asctime)s %(levelname)-5.5s [%(name)s] %(message)s" # Defaults for Collections cache_size: int max number loaded templates in collection 0 means unlimited loaded templates auto_reload: int min seconds to wait to check if needs reloading 0 means check on every rendering slurpy_directives: bool consume all whitespace trailing a directive # Defaults for Collections (and Templates) quoting: either("xml", "str", type) "xml" -> qpy.xml, "str" -> unicode input_encoding: str hint for how to decode template source filters: [callable] list of template post-evaluation filter functions """
""" domain public methods """ set_on_globals(name: str, value: any) -> None set_namespace_on_globals(name: either(str, None), obj: any, no_underscored: bool = True ) -> None get_collection(name: either(None, str, Collection) = None ) -> Collection set_collection(name: str, dir: str, cache_size: int = None, auto_reload: int = None, slurpy_directives: bool = None, quoting: either(str, type) = None, input_encoding: str = None, filters:[callable] = None ) -> None has_collection(name: str) -> bool """ methods wrapping collection.*_template() """" get_template(name, src=None, collection=None, raw=None, data=None, quoting=None, input_encoding=None, filters=None ) -> Template set_template(name, src=None, collection=None, raw=None, data=None, from_string=True, quoting=None, input_encoding=None, filters=None ) -> None has_template(name, collection=None) -> bool
Collection
class Collection(object): """ A collection of templates, rooted at a directory """ def __init__(self, domain, name, path, # defaults from Domain cache_size=None, auto_reload=None, slurpy_directives=None, # defaults (from Domain) for Templates quoting=None, input_encoding=None, filters=None): """ domain: either(None, Domain) name: str, name by which to retrieve the collection path: str, abs path for root folder for the collection For more on defaults from Domain init parameters, see the docstring in domain.Domain.__init__(). Preferred way to create a new collection: domain.set_collection() """
""" collection public methods """ get_template(name="", src=None, raw=None, data=None, quoting=None, input_encoding=None, filters=None ) -> Template """ get / load (if file-based) a template """ set_template(name, src, raw=None, data=None, from_string=True, quoting=None, input_encoding=None, filters=None ) -> None """ typically used to set a template from a string """ has_template(name) -> bool """ collection (read-only) attributes """ domain: Domain
Template
class Template(object): """ An Evoque template """ def __init__(self, domain, name, src=None, collection=None, raw=None, data=None, from_string=False, # defaults from Collection quoting=None, input_encoding=None, filters=None): """ domain: either(str, Domain) abs path, or actual domain instance name: str if no src this is collection-root-relative locator src: either(None, str) (file-based) collection-root-relative locator (from_string=True) template source string collection: either(None, str, Collection) None implies default collection, else str/Collection refer to an existing collection raw: bool if raw, do not compile the template source data: either(None, dict) default data for template evaluation from_string: bool when True src must specify the template source For more on defaults from Collection init parameters, see the docstring in collection.Collection.__init__(). Preferred way to create a new template: domain.{get_template(), set_template()} """
""" template public methods """ evoque(locals: dict = None, raw: bool = None, quoting: either(str, type) = None **kw) -> qsclass """ Render template, as an instance of `qsclass` (derived from `quoting` or from `template.qsclass`). If `locals` is None, use a copy of "own" locals. If `raw` or `template.raw` is True, return raw string without evaluation. Any `kw args` are set onto `locals` before evaluation. """ test() -> either(None, [qsclass]) """ Return a rendering per test directive """ unload() -> None """ Unloads self from collection """ """ template (read-only) attributes """ name: str collection: Collection raw: bool data: either(None, dict) # default data for template evaluation qsclass: either(unicode, qpy.xml, type) # quoted-no-more string class, determined from quoting init param input_encoding: str # encoding hint, default from collection filters: [callable] # post-evaluation filters, default from collection
Note there are no instance attributes that correspond
directly to the src, from_string and
quoting init parameters.
managing the collection’s template cache
Each collection controls its own cache, customizable
via two parameters that get their default
values from the domain:
cache_size: int = 0
sets the maximum number
of loaded templates in a collection, with a value of 0
meaning no limit.auto_reload: int = 60
sets the minimum number of seconds to wait before
doing astat
on the template file to see if it
has been modified. A value of 0 means to do the file system
check on each request for the template.
The collection cache is always on.
If you prefer to run like there was no cache, i.e. reloading
every template each time it is requested, you can set
cache_size=1
and auto_reload=0
.
If, on the other hand, you would like to run such that
templates are virtually never reloaded, you can set
cache_size=0
and
auto_reload=60*60*24
,
i.e. a sufficiently large number of seconds to wait
between checks.
template error handling
Template errors may occur in one of two contexts —
during loading or during evaluation.
It is of course possible that a loading error of one template
occurs during the evaluation of another, for example when a template
evoques another template that is not yet loaded.
Thus the big differentiator for errors is not the type of error
but when an error happens.
All errors not occurring during evaluation are raised as exceptions,
as any other normal application error.
The behaviour of evaluation errors may be controlled
via two domain parameters:
errors: int = 3
sets evaluation behaviour for errors, and may be an int between 0 and 4:4 = raise
will re-raise the exception given by the error.3 = render
will render the error as a 1-line descriptive summary of the error.2 = name
will render the error as the indicative string:
EvalError[expr]
, where
expr is the actual source of the
failed expression.1 = zero
(not yet implemented)
assume the zero object for the inferred python type
for the failed expression being evaluated. Examples,
if astr
then proceed with the empty string,
if alist
proceed with empty list,
if anint
proceed with 0, etc.0 = silent
no output is generated for the error,
i.e. errors are rendered as the the empty string.
log: logging.getLogger("evoque")
sets the logger to be used for all evaluation
errors;
independently of the setting oferrors
,
evaluation errors are always logged.
Also, the output of callinginspect(False)
from
within a template are sent to log.info().
restricted execution
Evoque supports
python expressions only. This means that
python statements are already naturally restricted as
when evaluated these will always raise a
SyntaxError
.
The problem of restricting execution is thus reduced
to the problem of making the eval
python built-in
safe. This is by no means an easy task, but at least with its
more limited scope it is simpler than attempting to
generically restrict or sandbox the python interpreter.
Running in restricted mode will (supposedly)
make it impossible to manipulate tangible resources,
such as files and sockets, from within a template.
Protection of intangible resources,
such as memory and CPU usage, is not (yet) provided (see below).
restricted: bool = False
This is the one domain init parameter that when set
to True
will initialize the domain to
run in restricted execution mode.
Setting this to True
has therefore the following
consequences:
1. Sets a dummy __builtins__
Sets a dummy __builtins__
empty dict on the
domain-wide globals dict used by eval.
2. No builtins that are deemed unsafe
For every builtin that is not deemed unsafe, will
add a top-level entry to the domain-wide globals dict used by eval.
Python (2.4, 2.5, 2.6, 3.0) __builtins__
currently considered unsafe
are (defined in the list Domain.DISALLOW_BUILTINS
):
DISALLOW_BUILTINS = ["_", "__debug__", "__doc__", "__import__", "__name__", "buffer", "callable", "classmethod", "coerce", "compile", "delattr", "dir", "eval", "execfile", "exit", "file", "getattr", "globals", "hasattr", "id", "input", "isinstance", "issubclass", "locals", "object", "open", "quit", "raw_input", "reload", "setattr", "staticmethod", "super", "type", "vars"]
In addition to the above, all subclasses of BaseException
(or, in Python 2.4, of Exception
) are considered
potentially unsafe and so are not made available
as entries on the globals dict.
This may seem overly restrictive, but then templating should not
require more than a small subset of python’s
possibilities, for convenience of doing simple progamming tasks
e.g. enumerating over a list.
3. Runtime scan of all expressions
At runtime, all string expressions to be evaluated,
necessarily via restrictedEvaluator[expression]
,
are first scanned for any disallowed attribute lookups,
and if a string expression matches, a LookupError
will be raised. The restricted_scan
pattern will match
any attempt to access any attribute that starts with one of the
following character sequences:
restricted_scan = re.compile(r"|\.\s*".join([ "__", "func_", "f_", "im_", "tb_", "gi_", "throw"]), re.DOTALL)
Note that expression-building trickery will not achieve anything,
as expressions are never evaluated twice i.e. an expression
may build a string but such a string cannot in anyway
be evaluated. To illustrate, consider an expression such as
${legit_obj.__subclasses__()}
.
This will fail
because it attempts an attribute lookup that starts with “__”.
Attempting to bypass this by doing something like
${"legit_obj." + "_"*2 + "subclasses" + "_"*2 + "()"}
will simply return the string
"legit_obj.__subclasses__()"
that cannot be re-rendered.
4. No re-evaluation of expressions
Evoque takes every precaution possible to make it impossible to
re-evaluate rendered results in any way.
For example, Evoque templates categorically cannot arbitrarily
set a variable, or initialize a new string-based template.
Furthermore, under restricted mode, builtins such as
eval
or getattr
are of course
not available.
Is this enough?
Depending on paranoia level, probably not.
Besides more testing to validate that the above effectively
does not allow the template programmer to access and manipulate
any tangible resources, it should still be possible to
bring down the interpreter via DOS maliciousness,
e.g. evaluating an expression for a very large
multiplication to consume all available memory, or to take
a very long time to finish.
Brett Cannon and Eric Wohlstadter, from UBC in Vancouver, have done
some thorough research into the feasibility of more ambitiously
applying a similar but more generalized approach to fully restrict
the python interpreter i.e. all possible expressions and statement
constructs —
they have summarized their work in this inspirational paper:
Controlling Access to Resources Within The Python Interpreter
Nonetheless, further runtime analysis of expressions will be required
to protect against intangible resources such as memory and
CPU usage, and how best to add such
protection is future work that is still to be done.
Thus, for the paranoid, restricted execution mode should for now
best be considered experimental.
managing the namespace
Besides turning
restricted mode on or off,
customization control of the evaluation context is
available by means of:
domain.set_on_globals(name: str, value: any) -> None domain.set_namespace_on_globals(name: either(str, None), obj: any, no_underscored:bool=True) -> None mylocals = dict() # prepared as necessary template.evoque(mylocals)
The domain methods allow adding of objects
onto the domain-wide globals dict.
no_underscored: bool = True
By default, the set_namespace_on_globals()
methods look for an __all__
attribute on
the source object, and if one is found will only set
the what is listed by it.
If no one is found, it will import all attributes
but by default not those starting with an underscore —
behaviour that may be changed by setting
no_underscored = False
.
In addution, passing a prepared mylocals
dict to
template.evoque()
can additionally expose
specific objects per template evoque’ation.
Howto
Because everything is sometimes not immediately obvious 😉
Sub-templates in markdown
This amounts to just specifying the markdown filter on the output of
the evoque’d template. It is easy to see how it works with an example
(coming straight from the unit test data included in the
Evoque distribution).
The template
markdown.html
$begin{my-markdown-template} ### From a markdown template! - item one - item two ${param} $end{my-markdown-template} \ <div class="markdown-output"> $evoque{#my-markdown-template, filters=[markdown], param="<xml/>"}\ </div>
For simplicity, we just use a single file, where the top-level template
evoques a locally-nested markdown template, but of course the source of the
sub-template could be anything you like.
Notice that all the outer template does is evoque the inner template
into a <div/>, specifying the filters to use as well as some
default data.
Output
We must supply the filter we would like to use, so we need to set it on the globals
of the evaluation namespace.
We can then render the template:
from markdown import markdown domain.set_on_globals("markdown", markdown) print domain.get_template("markdown.html").evoque()
That will produce the following output:
[KeyError: ‘auto’: File “/home/admin/py-packages/evoque/domain.py”, line 108, in get_collection
return self.collections[name]
: EvalError(evoque(name=u’markdown.txt’, collection=”auto”, raw=True, filters=[evoque_html_highlight]))]
Notice that automatic quoting of incoming data just happens
also for sub-templates in markdown — or in any other format.
Alternative – declaring preferred filters within template itself
A template may declare itself what filters should be used.
Sometimes we may not have write access to a template source, but in
those cases it would be easy to just read that source and wrap it into
a another template under our control. The template below
issues a $prefer{}
directive with
filters=[markdown]
as well as some default data
,
and gives identical results as above, but relieves the caller
from the need of specifying the filters or any default data on each call.
$begin{my-markdown-template} $prefer{ filters=[markdown], data=dict(param="<xml/>") } ### From a markdown template! - item one - item two ${param} $end{my-markdown-template} \ <div class="markdown-output"> $evoque{#my-markdown-template}\ </div>
Using only python callables
Just to get a little closer to what is going on underneath…
declaring filters
on the $evoque{}
call
or the $prefer{}
directive of a template is really just
a little syntactic sugar over doing everything
with pure python callables. Here’s what such an alternate version would
look like (here we are required to use the callable version of
the evoque
directive):
$begin{my-markdown-template} ### From a markdown template! - item one - item two ${param} $end{my-markdown-template} \ <div class="markdown-output"> ${xml(markdown(evoque("#my-markdown-template", param="<xml/>")))}\ </div>
This will produce the exact same output when rendered as above.
Sub-templates in reStructuredText
You can use
reStructuredText
or rst in a multitude of ways.
Typical usage from within a templating system is likely
that you want to build a web page that contains pieces of content
in rst i.e. you want to automatically process rst sub-templates
from within a containing HTML template —
to achieve this, all you need is to specify an appropriate
rst filter on the sub-template.
Let’s see this with an example (coming straight from the unit
test data included in the Evoque distribution).
The templates
First, just a sample rst source template file
(incidentally, this is the long description of the
QuickWikiEvoque package on PyPI ):
reStructuredText.rst
QuickWikiEvoque tutorial application - for Pylons_ >=${PYLONS_VER}, Evoque_ 0.4, SQLALchemy_ 0.5 An updated version of the original QuickWiki tutorial application for Pylons, to work with: - Pylons >=${PYLONS_VER} (instead of 0.9.6), - Evoque (instead of Mako) and - SQLAlchemy 0.5 (instead of 0.4). Please follow the original explanatory text at http://wiki.pylonshq.com/display/pylonsdocs/QuickWiki+Tutorial for step by step explanations, with the obvious replacements for the different versions as indicated above, and for the actual code just look at the source code in the distribution. .. _Pylons: http://www.pylonshq.com/ .. _Evoque: http://evoque.gizmojo.org/ .. _SQLAlchemy: http://www.sqlalchemy.org/
And a sample HTML containing template:
rst_container.html
<div class="rst"> $evoque{reStructuredText.rst, filters=[rst]} </div>
Output
We must define an appropriate rst-to-HTML filter
and the easiest way is to set it as a global on the domain.
We then render the containing template:
from docutils.core import publish_parts def rst(rst): return publish_parts(rst, writer_name="html")["fragment"] domain.set_on_globals("rst", rst) domain.get_template("rst_container.html").evoque(PYLONS_VER="0.9.7")
That will produce the following output (rendered as the content of the box below):
QuickWikiEvoque tutorial application –
for Pylons >=0.9.7, Evoque 0.4, SQLALchemy 0.5
An updated version of the original QuickWiki tutorial application
for Pylons, to work with:
- Pylons >=0.9.7 (instead of 0.9.6),
- Evoque (instead of Mako) and
- SQLAlchemy 0.5 (instead of 0.4).
Please follow the original explanatory text at
http://wiki.pylonshq.com/display/pylonsdocs/QuickWiki+Tutorial
for step by step explanations, with the obvious replacements for
the different versions as indicated above, and for the actual code
just look at the source code in the distribution.
Using a collection
When we have multiple rst templates we probably do not want to bother with
having to always set the same filters or other options each time we
evoque one of them.
For this case we can choose to define a collection to contain our rst
templates and set the filters or other options we want as defaults on the
collection. We may set this up simply with something like (here we just
specify only one default, filters, but collections support
several others):
domain.set_collection("rst", "/docs/templates/rst", filters=[rst])
Thus, any rst template we evoque from this collection
will inherit any defaults on the collection.
The price to be paid for this is that to address a template
from another collection, we need to explicitly specify
the target collection. So, for the example above, the outer
rst_container.html HTML template (that in this scenario
is not within the rst collection) would become:
<div class="rst"> $evoque{reStructuredText.rst, collection="rst"} </div>
Using without qpy and still doing html quoting
Qpy is only used when quoting="xml"
, thus if you run with
quoting="str"
you will not need to have Qpy installed, but you
will need to take care of quoting yourself.
As an example, let’s use the code from the the evoque_mq
benchmark that is included in the distribution:
First, make quoting="str"
the domain-wide default, and
we can also set, on the domain’s globals, the quoting function we want
to use:
from evoque.domain import Domain domain = Domain(/path/to/evoque/bench/evoque, quoting="str") import cgi domain.set_on_globals("quote", cgi.escape)
In the template itself template_mq.html
then specify explicitly the values that need to be quoted:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en"> <head> <title>${quote(title)}</title> </head> <body> $evoque{header_mq.html} $begin{greeting}<p>Hello, ${quote(you)}!</p>$end{greeting} $evoque{#greeting, you=user} $evoque{#greeting, you="me"} $evoque{#greeting, you="world"} <h2>Loop</h2> $if{items} <ul> $for{idx, item in enumerate(items)} <li${idx+1==len(items) and " class='last'" or ""}>${quote(item)}</li> $rof </ul> $fi $evoque{footer.html} $test{ title='Just a test', user='joe', items=['a', '<b/>', 'c'] } </body> </html>
Rendering is as usual, for example:
data = dict(title="Just a test", user="joe", items=["<n>%d</n>" % (num) for num in range(1, 15)]) domain.get_template("template_mq.html").evoque(data)
Defining custom quoted-no-more classes
A quoted-no-more infectious string type is a subclass of Python’s unicode
string class and instances of such a type are known to not need any further
quoting. It is infectious in the sense that when combined with
strings of other types, those strings will be first quoted.
This infectious behaviour may be summarized with (where qnm
is a
quoted-no-more instance and s
is some other string instance):
qnm + s = qnm + QNM(s)
qnm_join([s, qnm, ...]) = QNM(s) + qnm + ...
QNM("x %(key)s") % dict(key=s) = QNM("x ") + QNM(s)
Thus, the quoted-no-more pattern is a convenient way to liberally combine
strings without having to worry about whether they are quoted or not,
or whether they will ever be double-quoted.
The origins of the quoted-no-more pattern is the
QPY Quoted String Data Type, and it is central to Evoque —
for custom automatic once-and-only-once quoting, all templates need to do
is specify a quoted-no-more class as the value of the
quoting
parameter.
Custom quoted-no-more classes should subbclass
evoque.quoted.quoted_no_more
and provide an implementation for
the _quote(s)
class method, whose sole responsibility
is to quote a single unicode and return it as an instance of
the custom quoted-no-more class itself.
To explicitly quote a string or any other object, the class
method quote(obj)
is what should be used.
This class method is defined by the quoted_no_more
base class
and will internally call the _quote(s)
method on the
subclass — where s
is a unicode-ified representation of the
obj
parameter received by quote(obj)
.
The evoque.quoted
package includes some sample quoted-no-more
implementations, such as the ones for xml
and url
quoted strings.
example: xml
For demo purposes only, a Python implementation of the xml
quoted-no-more class is included in evoque.quoted.xml
:
__revision__ = "$Id$" from evoque.quoted.quoted_no_more import quoted_no_more class xml(quoted_no_more): """ xml instances are strings that require no further XML quoting NOTE: this python class is provided as a demo for how to define custom quoted-no-more classes in python. The QPY package provides an equivalent implementation of this class in C, at qpy.xml, that is much faster. """ def _quote(cls, s): """ (s:unicode) -> xml """ return cls(s.replace("&", "&" ).replace("<", "<" ).replace(">", ">" ).replace('"', """ ).replace("'", "'")) # ' is not HTML. _quote = classmethod(_quote) # generic join xml_join = xml().join __all__ = ["xml", "xml_join"]
To use this class, we just specify it as the value of the
quoting
parameter anywhere that it is supported
i.e. on the domain, on collections or on any individual template.
For example:
from evoque.quoted.xml import xml domain.get_template("template.html", quoting=xml).evoque()
Output the raw source of an evoque template
Just evoque()
with raw=True
and,
if we would like to escape html/xml source,
then also specify quoting="str"
. It does not matter how the evoque
template is defined, or what source format it is in. Here’s an example,
let’s first define a template template.html
:
$if{items} <ul> $for{ i, item in enumerate(items) } <li>${item}</li> $rof </ul> $fi
And we can evoque it as raw source from an application:
domain.get_template("template.html").evoque(raw=True, quoting="str")
Or, from another template in the same collection:
$evoque{template.html, raw=True, quoting="str"}
Additional raw source notes
Either of
get_template(raw=True).evoque()
or
get_template().evoque(raw=True)
is valid,
but there is a significant difference here:
stating get_template(raw=True)
will
(when template is not yet loaded) load it and not compile it;
stating evoque(raw=True)
will, after loading and compiling
the template if necessary, render the template’s raw source.
Similarly, it is possible to do either of
get_template(quoting="str").evoque()
or
get_template().evoque(quoting="str")
: the first, used when
the template is loaded, sets the template’s default
quoted-no-more string class to use, while the second specifies
which one to use for this evoque’ation.
Using the $evoque{template.html, raw=True, quoting="str"}
form, i.e. simultaneously loading and evoking a template from within
another template,
also has the implications that if the template is
not yet loaded then it will be
(a) loaded with unicode
as its
default quoted-no-more string class and
(b) loaded as a raw template, i.e. it will
not be compiled and any subsequent attempts of evoque(raw=False)
will result in an error.
Any template may be rendered raw.
Sometimes you know beforehand that a template will be almost always rendered raw.
In those cases it is more practical to specify a $prefer{raw=True}
directive once in the template itself, as opposed to specify same on each
evoque’ation of the template.
The same suggestion above applies also for quoting,
e.g. $prefer{quoting="str"}
.
Switching site template, dynamically, per user
The use of overlays, or template inheritance, where the
base template is used to define the site layout
could be a simple and effective way to manage a theme for a site.
In addition, multiple site themes could be managed simply via
multiple site base templates.
Evoque supports dynamic data-driven overlays —
and dynamic in two very handy ways:
-
First, by the possibility of using logically named templates that
allows assigning a name to any template, and then retrieving it by that name.
The dynamic part comes from the fact that we can at runtime change which
template is pointed to by a given name (by first doing a
template.unload()
and then loading another template
under the same name). -
Second, by the fact that a template may specify which
template it overlays by means of a parameter that will be evaluated for
each rendering at runtime.
For this case, the second technique makes it
really trivial to parametrize the base template per user.
Here’s the essence of an overlay page template that does this:
a dynamic page overlay: site_dyn_page_var.html
$overlay{name=my_site_theme} $begin{content}page: ${message}$end{content}
Where, on each rendering, which base template to use is determined
by the runtime value of my_site_theme
.
Assume we have loaded the template with:
page_overlay = domain.get_template("site_dyn_page_var.html")
We can then specify a different value for my_site_theme
on each rendering:
page_overlay.evoque(my_site_theme="site_template_divs.html", ... )
Note that if you want to make sure you always have a fallback value
for my_site_theme
(for all renderings, all templates, and all users)
then just set it on the domain’s globals
:
domain.set_on_globals("my_site_theme", "site_template_table.html")
example details
For completeness, let’s fill in the full details for this simple example
(that forms part of the Evoque test cases, and is included in the distribution).
a couple base templates as site themes
theme one: site_template_table.html
$begin{header}site-table: <h1>${title}</h1>$end{header} $begin{content}site-table: ${message}$end{content} $begin{footer}site-table: ${footer}$end{footer} <html> <head><title>site-table: ${title}</title></head> <body> <table class="layout"> <tr><th>$evoque{#header}</th></tr> <tr><td>$evoque{#content}</td></tr> <tr><td>$evoque{#footer}</td></tr> </table> </body> </html>
theme two: site_template_divs.html
$begin{header}site-div: <h1>${title}</h1>$end{header} $begin{content}site-div: ${message}$end{content} $begin{footer}site-div: ${footer}$end{footer} <html> <head><title>site-div: ${title}</title></head> <body> <div class="layout"> <div class="header">$evoque{#header}</div> <div class="content">$evoque{#content}</div> <div class="footer">$evoque{#footer}</div> </div> </body> </html>
a couple renderings of the same page overlay
using theme one:
page_overlay.evoque(my_site_theme="site_template_table.html", title="hey tabby!", message="you're kinda square!", footer="you reckon?")
gives the output:
[KeyError: ‘auto’: File “/home/admin/py-packages/evoque/domain.py”, line 108, in get_collection
return self.collections[name]
: EvalError(evoque(name=u’site_dyn_page_table.txt’, collection=”auto”, raw=True, filters=[evoque_html_highlight]))]
using theme two:
page_overlay.evoque(my_site_theme="site_template_divs.html", title="howdie!", message="ya bin free floatin' good?", footer="sure thang!")
gives the output:
[KeyError: ‘auto’: File “/home/admin/py-packages/evoque/domain.py”, line 108, in get_collection
return self.collections[name]
: EvalError(evoque(name=u’site_dyn_page_divs.txt’, collection=”auto”, raw=True, filters=[evoque_html_highlight]))]
Performance benchmarks
All performance benchmarks are included in the distribution
— please try out for yourself, the file
bench/README.txt
has some additional details for how to run each benchmark.
about the benchmarked systems
Qpy
is a templating system almost identical to straight
python and offering convenient and fast
(at least the C based version) html string building with
automatic escaping. From the Qpy readme:
Qpy provides a convenient mechanism for generating safely-quoted html
text from python code. It does this by implementing a quoted-no-more string
data type and a modification of the python compiler.
Amongst currently popular full-featured templating systems for python:
Mako is probably
the fastest pure-python text-based system;
Genshi is probably the
feature-richest XML-based system.
The performance of Evoque and Mako (comparable text-based systems)
seems to be more or less similar —
Evoque seems to be faster for general mixed-task templates,
while Mako seems to do better on loop-intensive templates.
Suggestions to make these benchmarks more relevant
are welcome, especially from anyone with particular
knowledge of the specific templating system in question.
platform
The actual times are coming off a
MacBook Pro with 2.4 GHz Intel Core 2 Duo, 2 GB of RAM,
and Mac OS X 10.5, running Python 2.6.1.
caution
Please remember that performance benchmarks are only relevant
when considered within an entire context, and they may vary
enormously between different combinations of hardware and software,
even if those difference may appear to be very slight.
In addition, two different systems
never quite do precisely the same thing,
however simple and apparently identical the timed task may be.
The numbers from these benchmarks are a tribute to python’s
conceptual integrity — that rewards a straightforward implementation
of a simple design with… amazing performance.
subs benchmark
For curiosity, a very simple performance look at doing
only string substitutions —
thus for this one we can even compare against
string.Template
from the python standard library.
We test a small template, mostly static text with just 6 variable
substitutions.
For each run, time is averaged over 2000 renderings, and the
best time of 4 runs is retained.
All times are in ms.
What you should want is automatic quoting and less time.
quoting | template.String | qpy 1.7 | evoque 0.4 | mako 0.2.4 | genshi_text 0.5.1 | |
---|---|---|---|---|---|---|
automatic | 0.028 | 0.034 | 0.112 | |||
automatic | R(a) | 0.038 | ||||
manual | 0.038 (b) | 0.107 | 0.169 | |||
none | 0.050 | 0.025 | 0.084 |
(a) Restricted —
qpy 1.7 /
mako 0.2.4 /
genshi 0.5.1
offer no support for restricted execution or sandboxing
(b) manual quoting implies no qpy
template and data
The (automatically quoted) evoque template.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <meta http-equiv="Content-Style-Type" content="text/css; charset=UTF-8" /> <meta http-equiv="imagetoolbar" content="no" /> <style type="text/css"> .signature { color: #977; font-weight: bold; } </style> <title>${title}</title> </head> <body> <p>Welcome back ${first}, you are logged in as </code>${username}<code> (last login: ${last}).</p> <p>Your balance is: ${balance}</p> <p>${comment}</p> </body> </html>
DATA = dict(title='Your balance', first="Joey", username='joe123', last="2008-02-29", balance=789.19, comment="Thank you <b>very</b> much!")
platform
The actual times are coming off a
MacBook Pro with 2.4 GHz Intel Core 2 Duo, 2 GB of RAM,
and Mac OS X 10.5, running Python 2.6.1.
caution
Please remember that performance benchmarks are only relevant
when considered within an entire context, and they may vary
enormously between different combinations of hardware and software,
even if those difference may appear to be very slight.
In addition, two different systems
never quite do precisely the same thing,
however simple and apparently identical the timed task may be.
basic benchmark
A basic benchmark, consisting of a small template
that does a variety of standard templating tasks such as looping,
calling nested and external sub-templates, escaping of data, etc.
For each run, time is averaged over 2000 renderings, and the
best time of 4 runs is retained.
All times are in ms.
What you should want is automatic quoting and less time.
quoting | qpy 1.7 | evoque 0.4 | mako 0.2.4 | genshi 0.5.1 | |
---|---|---|---|---|---|
automatic | 0.104 | 0.339 | 0.421 | 2.706 (c) | |
automatic | R(a) | 0.386 | |||
manual | 0.318 (b) | 0.387 | |||
none | 0.262 | 0.308 |
(a) Restricted —
qpy 1.7 /
mako 0.2.4 /
genshi 0.5.1
offer no support for restricted execution or sandboxing
(b) manual quoting implies no qpy
(c) xml mode
Inspiration for this benchmark is from the basic benchmark proposed
by Genshi.
template and data
The (automatically quoted) evoque templates.
template.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en"> <head> <title>${title}</title> </head> <body> $evoque{header.html} $begin{greeting}<p>Hello, ${you}!</p>$end{greeting} $evoque{#greeting, you=user} $evoque{#greeting, you="me"} $evoque{#greeting, you="world"} <h2>Loop</h2> $if{items} <ul> $for{idx, item in enumerate(items)} #[ An alternative to the expression below could also be an inlined if statement (that many templating systems are unable to do), such as: $if{idx+1==len(items)} class="last"$fi but it evaluates a little slower ]# <li${idx+1==len(items) and " class='last'" or ""}>${item}</li> $rof </ul> $fi $evoque{footer.html} $test{ title='Just a test', user='joe', items=['a', '<b/>', 'c'] } </body> </html>
header.html
<div id="header"> <h1>${title}</h1> </div>
footer.html
<div id="footer"></div>
DATA = dict(title='Just a test', user='joe', items=['<n>%d</n>' % (num) for num in range(1, 15)])
platform
The actual times are coming off a
MacBook Pro with 2.4 GHz Intel Core 2 Duo, 2 GB of RAM,
and Mac OS X 10.5, running Python 2.6.1.
caution
Please remember that performance benchmarks are only relevant
when considered within an entire context, and they may vary
enormously between different combinations of hardware and software,
even if those difference may appear to be very slight.
In addition, two different systems
never quite do precisely the same thing,
however simple and apparently identical the timed task may be.
bigtable benchmark
A simple brute force generation of a 10 columns x 1000 rows table,
inspired from the bigtable benchmark proposed by
Genshi.
For each run, time is averaged over 10 renderings, and the
best time of 4 runs is retained.
All times are in ms.
What you should want is automatic quoting and less time.
quoting | qpy 1.7 | evoque 0.4 | mako 0.2.4 | genshi 0.5.1 | |
---|---|---|---|---|---|
automatic | 30.75 | 52.69 | 30.16 | 376.61 (c) | |
automatic | R(a) | 61.80 | |||
manual | 51.65 (b) | 45.92 | |||
none | 30.72 | 13.30 | |||
none | T(d) | 7.77 | 7.36 |
(a) Restricted —
qpy 1.7 /
mako 0.2.4 /
genshi 0.5.1
offer no support for restricted execution or sandboxing
(b) manual quoting implies no qpy
(c) xml mode
(d) Tweaked — as close to pure python as possible, entire template is reduced to the single brute expression:
<table>${"".join(["<tr>%s</tr>" % "".join(["<td>%s</td>" % (col) for col in row]) for row in table])}</table>
template and data
The (automatically quoted) evoque template.
template_string = """<table> $for{ row in table } <tr>$for{ col in row }<td>${col}</td>$rof</tr> $rof </table> $test{ table=[("a","b","c","d","<escape-me/>","f","g","h","i","j")] } """
BASEROW = ("a","b","c","d","<escape-me/>","f","g","h","i","j") TABLE_DATA = [ BASEROW for x in range(1000) ]
platform
The actual times are coming off a
MacBook Pro with 2.4 GHz Intel Core 2 Duo, 2 GB of RAM,
and Mac OS X 10.5, running Python 2.6.1.
caution
Please remember that performance benchmarks are only relevant
when considered within an entire context, and they may vary
enormously between different combinations of hardware and software,
even if those difference may appear to be very slight.
In addition, two different systems
never quite do precisely the same thing,
however simple and apparently identical the timed task may be.
Frequently asked questions
Why yet another templating system?
For precisely the same reason as every other one, and a lot more!
You may wish to take a look at Evoque’s
list of distinguishing features.
Why the evoque name?
It is a triple trip tributing all at once
(a) the great little eval
python built-in,
(b) the (optional) Qpy third-party package that
not only guarantees that all data input is
always quoted but also that it is quoted once and only once,
and
(c) the act of evoking a document from a template and
a bunch of data.
Maybe these are more allusions than tributes,
possibly making all this seem a little cubist.
Don’t call me pablo,
just that, everything is related.
The three are but one.
Within a qpy.xml quoted template, how do I output escaped html?
By just declaring it unquoted, i.e. cast it to unicode.
E.g. if you want to output “<tag/>”
as part of the content in an html document, you can do
${unicode("<tag/>")}
. Of course instead of
the literal “<tag/>” string you can unicode()
anything you like, even the evoque’ations of any other template.
Do I have to install Qpy?
No, but if you don’t you will need to work harder to take advantage
of the killer feature of
automatic cross-site scripting protection
— by providing your own
custom quoted-no-more class.
Or, if all you want is just to take care of input quoting manually,
see the how to do manual quoting without qpy.
Why can’t I just do $variable instead of having to always specify ${variable}?
Evoque plays defensively. Anything immediately after the $ and before the
opening { is reserved for the name of directives, with ${ taken to imply
the start of an expression. This way, other directives could be added in
the future free of any backwards compatibility issues.
Is there something like a def
directive?
Yes. For evoque a def is just a sub-template, defined
with a begin/end
block.
Further nesting of sub-templates, i.e. sub of a sub etc, is
also supported. A sub-template defined anywhere may be addressed
from anywhere else.
How come there is no include
directive?
There is, it is called evoque
.
You can evoque any text file you like, or any other template or sub-template.
By default evoque’d files are evaluated as if they were templates — to
just include raw source evoque with raw=True
(see the raw source howto).
How come there is no rawtext
directive?
There is, it is called evoque
, with parameter raw=True
,
and if you want a different quoting you can also specify it e.g.
quoting="str"
.
You can evoque any template or sub-template,
e.g. any begin/end
block,
or any textual file.
See the raw source howto.
Why isn’t the prefer
directive just part of
the begin
directive?
Because a template need not have a
begin or an end
declared, but may still wish to define some
prefer parameters —
top-level templates are implicitly delineated
by the beginning and ending of a string
(typically the text content of a file) while
nested templates (that are first-class templates in their own right)
must be delineated with begin
and/or end
.
What about template inheritance?
Sure, Evoque supports template inheritance like no other,
with support even for
data-driven template inheritance.
See the overlay directive.
Is it possible to import
anything from within a template?
Allowing arbitrary python imports from directly within a template
will introduce a lot of difficult issues with restricted execution
mode. To support restricted mode,
a python application must retain some control on what may be added
to the evaluation context and it seems that the simplest way to
achieve this is to only allow the main application itself to affect
such additions, using the provided methods,
e.g. domain.set_namespace_on_globals()
.
See managing the namespace.
Why isn’t there an XML-compliant syntax?
Because Evoque is a generic text templating system, not an XML-based one.
However, there is conceptually no problem with providing an XML-compliant
syntax, e.g using processing instructions, for directives — this will
of course require a modified template parser.
Expressions, being already XML-compliant, will require no change.
I do not understand the attribute descriptions I see on this site,
for example
filters:[callable]
or file:either(None, str)
?
Those descriptions mimic the syntax that the
qp.lib.spec
module uses to actually
specify attribute allowed values and to perform the
corresponding runtime validation checks.
Here the clarity of the syntax is only used for documentation purposes.
The examples here mean that filters
is allowed to be a
list of callables (taking some liberty here, as callable
is not really a python type), and file
may be
either None
or an instance of str
.
If you would like to know more, here’s a
little write-up on the wonderful spec
module.
Or see the
spec.py source directly.
How do I specify the encoding of the output?
You don’t. It is always unicode.
Once you get the output, you can encode it as you please.
Can Evoque do all that other template systems are able to do?
Most probably, yes. It would help to know the specific task, in which case
I’ll be more than happy to help identify how it could be done with Evoque.
Integration with external systems
Descriptions, examples for using
Evoque Templating
with other systems.
django
Using Evoque Templating with
Django
is very simple. All that is needed is for a view
to return a Django HTTPResponse
object of an
evoque’d template. The commented code recipe below shows
one way how to set this up and how to use it. You would need
to copy the evoque_django.py
module into your
django project, and customize as necessary.
Why would you want to use Evoque with Django?
Besides being a full-featured template system, and a
legitimate contender for the
fastest pure-python text templating engine
(see benchmark),
it also offers some important features not offered
by other engines, such as
automatic input quoting and guaranteed XSS protection,
restricted template execution mode to be able to expose
your templates to untrusted editors, processing is always and only
done in unicode, etc.
For more, see features.
The code
evoque_django.py
import sys from os.path import join, dirname, abspath import logging from evoque.domain import Domain from django.http import HttpResponse # to add to global namespace from django.core.paginator import ObjectPaginator from my_site import formatting_utils # default template collection root directory DEFAULT_DIR = abspath(join(dirname(__file__), 'templates')) # create template domain instance # here restating all defaults, for doc convenience domain = Domain(DEFAULT_DIR, restricted=False, errors=3, log=logging.getLogger("evoque"), cache_size=0, auto_reload=60, slurpy_directives=True, quoting="xml", input_encoding="utf-8", filters=[] ) # adjust global namespace as needed by application domain.set_on_globals("ObjectPaginator", ObjectPaginator) domain.set_on_globals("site_name", "mysite.net") domain.set_namespace_on_globals("formatting_utils", formatting_utils) # set up any other collection, as needed by application # here restating all defaults -- when None, domain's value is used domain.set_collection("custom", "/home/me/custom_collection/", cache_size=None, auto_reload=None, slurpy_directives=None, quoting=None, input_encoding=None, filters=None) # here we would do any other domain setup we might need # e.g. set a template default for a given collection # for details see: http://evoque.gizmojo.org/usage/ # or the source of the evoque.domain module. def evoque_django_response(request, name, src=None, collection=None, raw=None, data=None, quoting=None, input_encoding=None, filters=None, **kwargs): """ Render an evoque template to a django response object """ # get the template, loading it if necessary. # only name is required but all supported options # are passed on for completeness. template = domain.get_template(name, src=src, collection=collection, raw=raw, data=data, quoting=quoting, input_encoding=input_encoding, filters=filters) # evoque the template and create a django HTTPResponse object: # - evoque'ations are always in unicode, so we decode to utf-8 # - any and all kwargs are added to locals namespace return HttpResponse(template.evoque(raw=raw, quoting=quoting, request=request, **kwargs).encode('utf-8'))
views.py
Just an indication of what you would need to add to your
views.py
:
from evoque_django import evoque_django_response def some_function_named_in_urlpatterns(request): return evoque_django_response(request, 'some_template.html', title="some title!")
In addition to the title
kwarg we can of course
specify any others, as needed by the view. They will be transferred to the
template’s locals evaluation dictionary.
It is important to remember that an Evoque template collection opens up
access to all files below the specified folder root. Thus you may choose to
organize the templates hierarchically e.g. we could rename and place some_template.html
to some_function/template.html
(always relative to the collection’s root folder).
We can then specify to use it in this way:
evoque_django_response(request, 'some_function/template.html')
If for some reason we would wish to use a template that is not
part of the default collection, we would just need to specify
the collection e.g.
evoque_django_response(request, 'some_template.html', collection="custom")
gz
Gizmo(QP) is an extension of the
QP Web Framework.
Work on sites made with Gizmo(QP) has been one of the
primary motivator for the design choices that went into Evoque.
Thus Gizmo(QP) (version >= 0.8) naturally supports Evoque.
Current examples of integration of Evoque with Gizmo(QP) are the
online demos for Gizmo(QP).
pylons
Pylons encourages a good separation
between the model, the controllers, and the views. It also is very
flexible about using different template engines for different views.
This note discusses a way for how to set and use
Evoque Templating as the default template engine in
a Pylons (minimum version 0.9.7) application.
You may also wish to explore the
QuickWikiEvoque
tutorial application, that offers a complete Pylons 0.9.7 + SQLAlchemy 0.5
application that you can downlaod and play with.
Let’s assume we create a site from scratch, let’s call it evoque_site.
When asked, enter the template_engine as follows:
Selected and implied templates:
Pylons#pylons Pylons application template
Variables:
egg: evoque_site
package: evoque_site
project: evoque_site
Enter template_engine (mako/genshi/jinja/etc: Template language) [‘mako’]: evoque
Enter sqlalchemy (True/False: Include SQLAlchemy 0.4 configuration) [False]:
adjust application files
We will then need to add the following codes to the files/functions indicated:
evoque_site.config.environment.load_environment(global_conf, app_conf)
# Evoque Templating: # http://evoque.gizmojo.org/ext/pylons/ # http://evoque.gizmojo.org/usage/api/ import logging from evoque.domain import Domain from webhelpers.html import literal evoque_domain = Domain( # root folder for the default template collection, must be abspath; # as default use pylon's first templates folder os.path.join(root, app_conf.get("evoque.default_dir") or paths['templates'][0]), # whether evaluation namespace is restricted or not restricted = app_conf.get("evoque.restricted")=="true", # how should any evaluation errors be rendered # int 0 to 4, for: [silent, zero, name, render, raise] errors = int(app_conf.get("evoque.errors", 3)), # evoque logger; additional setings should be specified via the pylons # config ini file, just as for any other logger in a pylons application log = logging.getLogger("evoque"), # [collections] int, max loaded templates in a collection cache_size = int(app_conf.get("evoque.cache_size", 0)), # [collections] int, min seconds to wait between checks for # whether a template needs reloading auto_reload = int(app_conf.get("evoque.auto_reload", 60)), # [collections] bool, consume all whitespace trailing a directive slurpy_directives = app_conf.get("evoque.slurpy_directives")=="true", # [collections/templates] str or class, to specify the *escaped* # string class that should be used i.e. if any str input is not of # this type, then cast it to this type). # Builtin str key values are: "xml" -> qpy.xml, "str" -> unicode # In a Pylons context we use the webhelpers.html.literal class quoting = app_conf.get("evoque.quoting", literal), # [collections/templates] str, preferred encoding to be tried # first when decoding template source. Evoque decodes templates # strings heuristically, i.e. guesses the input encoding. input_encoding = app_conf.get("evoque.input_encoding", "utf-8"), # [collections/templates] list of filter functions, each having # the following signature: filter_func(s:basestring) -> basestring # The functions will be called, in a left-to-right order, after # template is rendered. NOTE: not settable from the conf ini. filters=[] ) # default template (optional) i.e. what to receive # when calling domain.get_template() with no params if app_conf.get("evoque.default_template"): evoque_domain.get_collection().get_template("", src=app_conf.get("evoque.default_template")) # attach evoque domain to app_globals config['pylons.app_globals'].evoque_domain = evoque_domain
development.ini
The above config load_environment()
code may be
customized on a per-deployment basis, by specifying any of the
following parameters under the [app:main]
section of the the deployment’s .ini
file:
# Evoque Domain : http://evoque.gizmojo.org/ext/pylons/ evoque.default_dir = evoque.default_template = evoque.restricted = false evoque.errors = 3 evoque.cache_size = 0 evoque.auto_reload = 2 evoque.slurpy_directives = true evoque.quoting = evoque.input_encoding = utf-8
If evoque.default_dir
is not set,
then Pylon’s first templates folder is used.
The filters
parameter (on Domain it only serves as site-wide default)
is not settable in the ini
.
For a representative usage example of filters
see the
Sub-templates in markdown howto.
logging.getLogger(“evoque”)
Just like any other logger used in a Pylons application, the
evoque
logger may be adjusted via a deployment’s
conf ini file.
evoque_site/lib/base.py
from pylons.templating import pylons_globals, cached_template def render_evoque(template_name, extra_vars=None, cache_key=None, cache_type=None, cache_expire=None, collection=None, raw=False, quoting=None): """ Render a template with Evoque : http://evoque.gizmojo.org/ext/pylons/ Accepts the cache options ``cache_key``, ``cache_type``, and ``cache_expire``. The remaining options affect how template is located, loaded and rendered: - template_name: normally this is the collection-root-relative locator for the template; if template had been previously with location specified via the src option, then may be any arbitrary string identifier. - collection: either(None, str, Collection), an existing collection, None implies the default collection - raw: bool, render the raw template source - quoting: either(str, type), sets the Quoted-No-More string class to use, None uses the collection's default, "xml" uses qpy.xml, "str" uses unicode """ # Create a render callable for the cache function def render_template(): # Get the globals # Note: when running in restricted mode which of these globals are # to be exposed should be a lot more selective. pg = pylons_globals() # Update any extra vars if needed if extra_vars: pg.update(extra_vars) # Grab a template reference template = pg['app_globals'].evoque_domain.get_template( template_name, collection=collection, raw=raw, quoting=quoting) return template.evoque(pg, raw=raw, quoting=quoting) return cached_template(template_name, render_template, cache_key=cache_key, cache_type=cache_type, cache_expire=cache_expire) render = render_evoque
evoque_site/setup.py
install_requires=[ "Pylons>=0.9.7", ..., "evoque>=0.4" ]
add a controller and template
Let’s first add and edit a simple controller
and then add a template just to test:
evoque_site/controllers/hello.py
class HelloController(BaseController): def index(self): # Return a rendered template with evoque c.title = "Hello from evoque!" return render("template.html")
evoque_site/templates/template.html
<html> <head><title>evoque + pylons : ${c.title}</title></head> <style type="text/css"> div.code { white-space: pre; font-family: monospace; line-height: 1.1em; padding: 0.6em; margin-bottom: 2em; border: 1px solid #ccc; background-color: #fafafe; } </style> <body> <h2>${c.title}</h2> <h3>template evaluation context</h3> <div class="code">${inspect()}</div> </body> </html>
Define a route for the hello.index()
view, and load it
in your web browser.
You should see a simple page that dumps the template evaluation context.
qp
Integrating Evoque Templating with the
QP Web Framework
is pretty straightforward. As an explanation, here’s the echo
demo site, included in the QP distribution, modified to use Evoque.
The changes are probably self-explanatory, but for the record
here’s a commentary on each one:
- First of all
slash.qpy
is renamed toslash.py
as the source code is now 100% python. - We import (first 2 import lines) what we need to be able to
set up an Evoquedomain
. - The functions
qp.pub.common.header
and
qp.pub.common.footer
are no longer needed, so are not imported.
Instead, we do importqp.pub.common.page
, and the template used
for each page rendering will take care of its own header and footer. - We override
Publisher.__init__()
to be able to
set up an Evoquedomain
for the site, via the
separately specifiedset_domain()
method.-
We designate the
evoque/
sub-folder
as root for our default template collection,
but this can be anywhere on the file system.
We may of course also specify other collections. -
We extend the evaluation global namespace as we need to,
here we only will need thepformat
utility. - We set a template under a name that we will use as default.
-
We designate the
- We override
Publisher.page()
so that it knows how
to get and evoque a template. Note that thepage()
API
remains identical — except for the added optionaltemplate
keyword by which an export may select which template to use.
If notemplate
keyword is specified on
page()
then the default template name is used. - We migrate the presentation code from
SiteDirectory.index()
to theitems.html
template. We also tell the
index()
method to use theitems.html
template. - Just for the heck of it, we also make
items.html
anoverlay
,
overriding only thecontent
sub-template,
on top ofbase.html
—
that takes over the functionality of
theheader()
andfooter()
QP functions
while also offering additional flexibility, e.g. may have as many
base templates as desired, or a base template can be more complex
than just to handle a header and a footer.
The code – slash.py
import sys from os.path import join, dirname, abspath import logging from evoque.domain import Domain from qp.pub.publish import Publisher from qp.pub.common import get_request, page from qp.fill.directory import Directory from pprint import pformat class SitePublisher(Publisher): configuration = dict( http_address=('', 8001), as_https_address=('localhost', 9001), https_address=('localhost', 10001), scgi_address=('localhost', 11001), ) def __init__(self, **kwargs): super(SitePublisher, self).__init__(**kwargs) self.set_domain() def set_domain(self): default_dir = abspath(join(dirname(__file__), 'evoque')) # create template domain instance # here restating all defaults, for doc convenience self.domain = Domain(default_dir, restricted=False, errors=3, log=logging.getLogger("evoque"), cache_size=0, auto_reload=60, slurpy_directives=True, quoting="xml", input_encoding="utf-8", filters=[] ) # extensions to global namespace self.domain.set_on_globals("pformat", pformat) # preload a default template, e.g. for error pages self.domain.get_collection().get_template("", "base.html") # other setup e.g. adding other collections, template aliases # see: http://evoque.gizmojo.org/usage/ def page(self, title, *content, **kwargs): """(title, *content, **kwargs) -> qpy.xml Return a page formatted according to the site-standard. """ # we make use of only the "template" kwarg -- there are # other possibilities, see: domain.get_template() template = kwargs.get("template", "") return self.domain.get_template(template).evoque( title=title, content=content, **kwargs) class SiteDirectory(Directory): """ This site displays the http request. """ def get_exports(self): yield '', 'index', None, None def index(self): items = get_request().__dict__.items() items.sort() return page('HTTP Request', template="items.html", items=items) def _q_traverse(self, components): return self.index()
The templates
base.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <head> <title>${title}</title> </head> <body class="standard"> $begin{content} ${content} $end{content} $evoque{#content} </body> </html>
items.html
$overlay{base.html} $begin{content} <h1>${title}</h1> <div style="margin:1em; padding:1em; border:1px solid gray;"> <dl> $for{ key, value in items } <dt>${key}</dt> <dd style="white-space:pre">${pformat(value)}</dd> $rof </dl> </div> $end{content}
werkzeug
Werkzeug is a collection
of WSGI utility modules.
The werkzeug web site
presents (on welcome page click on nutshell to see the code)
a very simple demo application, that is described as:
A tiny WSGI application (about 50 lines of code) that
just uses werkzeug’s routing system and uses template names as
endpoints. These templates are then loaded with Mako,
rendered and returned as responses.
It is really trivial to make an equivalent application that uses
Evoque instead. The changes to the two concerned files
werkzeug_nutshell.py
and
say_hello.html
, shown further below,
are superficial:
- Initialize an Evoque
domain
, thus implicitly also defining
the default template collection. - Evoque’ing the template is identical, except for a different method name.
- In the sample
say_hello.html
template, remove the unnecessary
h
filter (Evoque automatically quotes all incoming unquoted data). - Make the sample
say_hello.html
template a valid HTML document 😉
werkzeug_nutshell.py
!/usr/bin/env python # -*- coding: utf-8 -*- from os import path from werkzeug import BaseRequest, BaseResponse, \ SharedDataMiddleware, responder from werkzeug.routing import Map, Rule from werkzeug.exceptions import HTTPException from evoque.domain import Domain # path for this file's folder # used as root for default template collection root_path = path.abspath(path.dirname(__file__)) # create an evoque template domain (and default collection) domain = Domain(root_path) # create the URL map. The endpoints are template names without the # .html extension, to be rendered by the `dispatch_request` function. url_map = Map([ Rule('/', endpoint='index'), Rule('/hello/', defaults={'name': 'World'}, endpoint='say_hello'), Rule('/hello/<name>', endpoint='say_hello'), Rule('/shared/<file>', endpoint='shared', build_only=True) ]) def dispatch_request(environ, start_response): """ A simple dispatch function that (if decorated with responder) is the complete WSGI application that does the template rendering and error handling. """ # first we bind the url map to the current request adapter = url_map.bind_to_environ(environ) # then we wrap all the calls in a try/except for HTTP exceptions try: # get the endpoint and the values (variable or parts) # of the adapter. If the match fails it raises a NotFound # exception which is a HTTPException which we catch endpoint, values = adapter.match() # create a new request object for the incoming WSGI environ request = BaseRequest(environ) # create an empty response object with the correct mimetype. response = BaseResponse(mimetype='text/html') # get the template and render it. Pass some useful stuff to # the template (request and response objects, the current # url endpoint, the url values and a url_for function which # can be used to generate urls template = domain.get_template(endpoint + '.html') response.write(template.evoque( request=request, response=response, endpoint=endpoint, url_for=lambda e, **v: adapter.build(e, v), url_values=values )) # return the response return response except HTTPException, e: # if an http exception is caught we can return it as # response because those exceptions render standard error # messages under wsgi return e # finish the wsgi application by wrapping it in a middleware that # serves static files in the shared folder and wrap the dispatch # function in a responder so that the return value is called as # wsgi application. application = SharedDataMiddleware(responder(dispatch_request), { '/shared': path.join(root_path, 'shared') }) # if the script is called from the command line start the # application with the development server on localhost:4000 if __name__ == '__main__': from werkzeug import run_simple run_simple('localhost', 4000, application)
say_hello.html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html><head> <title>Hello World Fun!</title> </head><body> <h1>Hello ${url_values['name']}!</h1> <p><a href="${url_for('index')}">back to the index</a></p> </body></html>
Under consideration
For generation of XML documents,
add filters for validating the generated output
for both document (page) and fragment levels.
Should be easy to turn on and off, e.g. for development or for deployment.
Support for expression and directive level caching.
A quick test shows it can speed up rendering times by order 10 or greater
in heavily repetitive cases.
A general syntax (that may be applied equally to both directives and expressions)
could be something like:
where
condition
is a boolean expression to be evaluated at
template compile-time, and will determine whether the
caching is applied or not e.g. for development or production scenarios,
and *keys
are simply keys of items (or, more generally,
just arbitrary string expressions to be evaluated in the evaluation
namespace) and will be used to generate a cache_key
to represents the desired state as per the context.
Such a syntax can be applied identically also to expressions i.e. when no
directive is present e.g.
For the implementation a caching library such as
Beaker
could be used, or an own simple memory based scheme adopted
(a la collection.cache).
Depending on the implementation,
other general expression caching options
e.g. cache_type=”memory”,
could be exposed as **kwargs on cache().
There should not be any performance penalty for when
the cache is inactive i.e. the cache check should be
compiled out of the loaded template.
A cache()
callable will be made available
by default on the evaluation context.
Conditional expression-level decorators.
Generalize the cache idea above, to provide an open-ended
mechanism to conditionally capture output of any dir/expr and
do something with it — but with the possibility of turning
it on and off depending on context.
E.g. for the XML fragment well-formedness validation, one could do:
that will, when the dev
evaluates to True
,
intercept the output and do the validation check,
raising an error on failure. On the other hand,
when dev
evaluates to False
,
the xml_validate_wf()
interception will be compiled out
of the loaded template.
Applications would also be able to add any such custom decorator
to their template evaluation context.
Cache the compiled template on disk,
for the benefit of faster startup times,
appreciable for usage in a cgi environment.
Introspection into the collections and their
respective caches of loaded templates.
Consider supporting a more convenient way to specify filters
on expressions, e.g. ${ expr | url, trim, etc }
would do the same thing as the current
${etc(trim(url(expr)))}
.
Improve ability to “extract” pieces of
text from files as the source of a template,
without needing that those files be touched in any way.
Criteria could be pairs of regular expressions, line numbers,
format-specific parsing directives, combination of criteria, etc.
Could be implemented as an
extension to the evoque()
built-in callable,
or as a new callable such as extract()
.
done
Add documentation for defining custom quoted-no-more classes.
Add sample implementations e.g. for url.
Support also for python 3.0,
maybe from the same Evoque python 2.5 code base.
See the changelog.
Evoque copyright and license
Change log
0.x
- Added method: evoque.collection.Cache.clear()
-
Fixed a locals data dict isolation bug in recursive template calls (a
template directly evoking itself). -
Added evoque.quoted package, containing a quoted-no-more base class
to facilitate defining custom quoted-no-more classes. Added sample
quoted-no-more class implementations for xml and url. -
Changed the default quoted-no-more class used within a Pylons context,
to webhelpers.html.literal i.e. the evoque Domain is initialized with:
quoting = app_conf.get(“evoque.quoting”, webhelpers.html.literal) -
Added module utils.pyglexers, containing pygments lexers:
EvoqueLexer, EvoqueHtmlLexer, EvoqueXmlLexer
0.4 (2009-01-21):
-
Evoque (package/unittests) now runs on all of python 2.4, 2.5, 2.6 and 3.0,
simultaneously from the same identical code base. -
The decodeh module (encoding guessing algorithm, now even more important in
py3 when opening text files) becomes py3-aware by:- making read_from_file() always opens a file in binary mode
- handling bytes as the non-unicode str case in py3
-
Tighter lockdown of the in-process sandbox (thanks to Daniel Diniz for
feedback and additional testing) by:-
fail for all expressions that include lookups for attributes starting
with any of: “__”, “func_”, “f_”, “im_”, “tb_”, “gi_”, “throw” - added “object” to the disallowed list of builtins
-
fail for all expressions that include lookups for attributes starting
-
Changed behaviour of the Domain init parameter slurpy_directives=True, that
now behaves in the following way: for each directive, strip all
leading/trailing whitespace on the same line plus (on left) the initial
newline — if and only if the end of the line being slurped is made up of
only whitespace. - Added whitespace test case.
- Updated to qpy 1.7.
-
In the unit tests, replaced hash(s) as a shorthand way to compare strings
with explicit strings. -
Corrected problem with yield and try/finally that was breaking
Template.test() under python 2.4. -
In restricted mode, when initializing the globals namespace under
python 2.4, the issubclass check is done against Exception as opposed
to BaseException (that is not available in 2.4).
0.3 (2008-09-06):
-
All overlay keyword parameters may now be evaluated at runtime (in the same
way as for the evoque directive); in addition to the first “name” parameter,
the other parameters of the overlay directive [src, collection, space] may
now also be set to a variable for evaluation at runtime. -
Runtime evaluation of the overlay directive now explicitly raises a
SyntaxError if a keyword parameter other than [name, src, collection, space]
is specified. -
Added labels, eval_overlay attributes to Template; removed overlay dict
as attribute, replaced with template.get_space_overlay() that dynamically
returns a (space:bool, overlay:dict) tuple on each call, where the returned
overlay dict contains only parameters needed for calling get_template(). -
Renamed Template methods reload() and reload_from_string() to
decode() and decode_from_string(). -
Added possibility to specify a “local” src to $evoque{} when addressing a
template that is (a) named (b) file-based (c) nested and (d) within the
same collection; such src values must start with “#” and the calling
template’s file is used to normalize the src
(see Evaluator.evoque()). -
Refined the check to determine whether a KeyError during evaluation is
really coming from the evaluator’s lookup or from further down. -
Changed Template.test() method to be a generator of template evaluations,
thus resulting in that any successful evaluations prior to a first failure
are returned.
0.2 (2008-08-04):
-
The name first parameter of the evoque and overlay directives i.e.
$evoque{name} and $overlay{name} may now be specified in one of the
following ways:- an unquoted str arg : interpreted literally
- a quoted str arg : interpreted literally
-
name=an unquoted str : str is interpreted as variable name, and
evaluated at runtime within the template’s context - name=a quoted str : interpreted literally
- Template.unload() now recurses and unloads also any local sub-templates.
- Adopted default logging to be standard python logger for “evoque”.
-
Changed to True as default value of the “output” parameter of the
builtin callable ${inspect(output=True)}. -
Rendering of templates with raw=True now correctly processes any specified
post-evaluation filters.
0.1.2 (2008-07-27):
-
Added ext/pylons for integrating Evoque with Pylons, both as documentation
(on web site) and as sample code (in distribution). -
Corrected problem with auto-refreshing of a template with a modified value
for “data” in an embedded $prefer{data=dict()} directive. -
Removed unused methods on Template:
set_locals(), set_on_locals(), set_namespace_on_locals() -
Improved benchmark suite, in particular added benchmarks for mako in
automatic quoting mode. -
decodeh.py: added catch of ValueError from locale.getdefaultlocale(), to be
able to ignore exceptions of the form “ValueError: unknown locale: XXX” when
the LANG or any LC_* env variables are set to an unknown locale. -
decodeh.py: added global IGNORE_ENCS list, and added the non-existent “cp0”
encoding that is sometimes returned by Windows, thus avoiding a
“LookupError: unknown encoding: cp0” further downstream.
0.1.1 (2008-03-23):
-
Fixed bug with top-level middle text (in between directive blocks) not
being parsed, see: test_mid_text(). - Some minor changes to parsing code, for better robustness and clarity.
- Added ext/django recipe, for using evoque with django.
- Added ext/qp/echo sample qp application.
- Added ext/werkzeug sample WSGI application.
0.1 (2008-03-07):
- First public release.
Download and support
Single-page documentation
For printing convenience, this entire site may also be evoque’d
as a single page:
Typo reports, suggestions, observations
are appreciated!
Download Evoque version 0.4
(2009-01-21)
You may wish to take a look at the
change log
for this and previous releases.
It is also highly recommended to download and install the
Qpy
unicode templating utility that provides the qpy.xml
Quoted-No-More class for automatic input escaping.
You may prefer to use a command line client such as wget
to download these, as indicated below.
Installation
To install, you may either use easy_install
or
standard python distutils
(i.e. download, unpack, install) as detailed below:
easy_install evoque
easy_install qpy # optional, recommended
# Installing with distutils
wget https://gizmojo.org/dist/evoque-0.4.tar.gz
tar zxvf evoque-0.4.tar.gz
cd evoque-0.4
python setup.py install
cd .. # qpy, optional, recommended
wget http://www.mems-exchange.org/software/files/qpy/qpy-1.8.tar.gz
tar xzf qpy-1.8.tar.gz
cd qpy-1.8
python setup.py install
# Test
python evoque-0.4/test/test_evoque.py
Acknowledgements
Thank you to
MEMS and Nanotechnology Exchange
for the
great software
they have made available to the python community — the
Qpy package
that is used by Evoque is one of these packages.
In particular, a thank you to David Binger for his intense and
infectious appreciation of simplicity, and therefore complexity.
Thank you to Skip Montanaro and to Martin v. Löwis
for their help with the
decodeh module,
now part of Evoque.
The physical source lines of code (SLOC) count is generated using
SLOCCount
by David A. Wheeler.
Steven Degraeve for
the heading area background banner on the pages of this site.
The multitude of templating systems for python over the past years
have contributed to the clarification of the ideas and priorities
for Evoque. It is impossible to identify which and what —
a big thank you to the python community in general for
the openness and sharing of innumerable ideas
and disinterested hours of work.
Thank you to Alex Martelli for his help and collaboration
on the development of
XYAPTU: Lightweight XML/HTML Document Template Engine for Python.
This single-module templating system, written back in 2002,
is the beginning of the ideas that have evolved over the years
since then and that are now manifest in Evoque.