backslashes get stripped from $ signs

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

backslashes get stripped from $ signs

Dieter Plaetinck
Hi guys,
using pyblosxom 1.5 from git
with an very simple entry like the following:

$ cat entries/backslash.txt
backslash
PS1="\u@\h \[\$foo \]\W\[$RST\] \$"
<![CDATA[
PS1="\u@\h \[\$foo \]\W\[$RST\] \$"
]]>

PB emits html code as follows:
(...)
PS1="\u@\h \[$foo \]\W\[$RST\] $"
<![CDATA[
PS1="\u@\h \[$foo \]\W\[$RST\] $"
]]>
(...)

notice how the backslashes before the $'s are now gone.
i disabled *all* plugins and use the default (plain) parser.
blog-encoding is utf-8
I have this issue with both static rendering and "normal mode".
I use a regular html flavour and can reproduce it even like this:

$ cat flavours/html.flav/story
$(body)


anyone an idea?

------------------------------------------------------------------------------
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and,
should the need arise, upgrade to a full multi-node Oracle RAC database
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
pyblosxom-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/pyblosxom-users
Reply | Threaded
Open this post in threaded view
|

Re: backslashes get stripped from $ signs

will kahn-greene
Looks like a problem with the blosxom renderer.  Templates use $ for
variables.  It allows you to escape the $ in a template with a \ so that
you can have dollar signs show up in templates.

However, it looks like render_template totally does this wrong by
changing \$ to $ after the variables have been expanded and affecting
the variable expansion values.  Thus any \$ get converted to $ in the
story body.

The code in the blosxom renderer that handles variable finding and
substitution is pretty complex.  Seems like it might be a good idea to
ditch that code, switch to using a real parser, and then we'd be able to
trivially fix this issue, too.  We'd also be able to add some other
features like if blocks.

I don't think that's something I want to do in 1.5, though.  This is
definitely a 1.6 issue.

In the meantime, you could add a second \ or patch your copy of the
renderer locally.


On 12/26/2010 05:25 PM, Dieter Plaetinck wrote:

> Hi guys,
> using pyblosxom 1.5 from git
> with an very simple entry like the following:
>
> $ cat entries/backslash.txt
> backslash
> PS1="\u@\h \[\$foo \]\W\[$RST\] \$"
> <![CDATA[
> PS1="\u@\h \[\$foo \]\W\[$RST\] \$"
> ]]>
>
> PB emits html code as follows:
> (...)
> PS1="\u@\h \[$foo \]\W\[$RST\] $"
> <![CDATA[
> PS1="\u@\h \[$foo \]\W\[$RST\] $"
> ]]>
> (...)
>
> notice how the backslashes before the $'s are now gone.
> i disabled *all* plugins and use the default (plain) parser.
> blog-encoding is utf-8
> I have this issue with both static rendering and "normal mode".
> I use a regular html flavour and can reproduce it even like this:
>
> $ cat flavours/html.flav/story
> $(body)
>
>
> anyone an idea?
>
> ------------------------------------------------------------------------------
> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
> to consolidate database storage, standardize their database environment, and,
> should the need arise, upgrade to a full multi-node Oracle RAC database
> without downtime or disruption
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> pyblosxom-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/pyblosxom-users
>

------------------------------------------------------------------------------
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and,
should the need arise, upgrade to a full multi-node Oracle RAC database
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
pyblosxom-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/pyblosxom-users
Reply | Threaded
Open this post in threaded view
|

Re: backslashes get stripped from $ signs

Dieter Plaetinck
On Sun, 26 Dec 2010 20:41:16 -0500
will kahn-greene <[hidden email]> wrote:

> Looks like a problem with the blosxom renderer.  Templates use $ for
> variables.  It allows you to escape the $ in a template with a \ so
> that you can have dollar signs show up in templates.
>
> However, it looks like render_template totally does this wrong by
> changing \$ to $ after the variables have been expanded and affecting
> the variable expansion values.  Thus any \$ get converted to $ in the
> story body.
>
> The code in the blosxom renderer that handles variable finding and
> substitution is pretty complex.  Seems like it might be a good idea to
> ditch that code, switch to using a real parser, and then we'd be able
> to trivially fix this issue, too.  We'd also be able to add some other
> features like if blocks.
>
> I don't think that's something I want to do in 1.5, though.  This is
> definitely a 1.6 issue.

fair enough, once we have the new bugtracker in place we can start
gathering requirements to base the decision of the new template engine
 
> In the meantime, you could add a second \ or patch your copy of the
> renderer locally.

thanks, i attempted patching, but failed:

diff --git a/Pyblosxom/tools.py b/Pyblosxom/tools.py
index c560bef..f24b510 100644
--- a/Pyblosxom/tools.py
+++ b/Pyblosxom/tools.py
@@ -482,6 +482,10 @@ def parse(request, var_dict, template):
     :returns: the template string with template variables expanded.
     """
     encoding = request.config.get("blog_encoding", "utf-8")
+    # after the expansion, \$ in templates will be rendered as $, but this will get applied
+    # to the entire output.  To protect expanded content from this, we double escape first
+    if 'body' in var_dict.keys():
+        var_dict['body'] = var_dict['body'].replace(r'\$', '\\\$')
     replacer = Replacer(request, encoding, var_dict)
     return _VAR_REGEXP.sub(replacer.replace, template)
 

this kind of works, except that for some reason i need 3 backslashes in the replacement, and this will also change '$' into '\$',
which i don't want, obviously. changing the first param to '\\$' doesn't help either.  I'm not sure what the 'r' is for, but I saw it
in BlosxomRenderer:render_template(), and using it doesn't seem to change anything.

Dieter

------------------------------------------------------------------------------
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and,
should the need arise, upgrade to a full multi-node Oracle RAC database
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
pyblosxom-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/pyblosxom-users
Reply | Threaded
Open this post in threaded view
|

Re: backslashes get stripped from $ signs

Dieter Plaetinck
On Mon, 27 Dec 2010 11:46:48 +0100
Dieter Plaetinck <[hidden email]> wrote:

> On Sun, 26 Dec 2010 20:41:16 -0500
> will kahn-greene <[hidden email]> wrote:
>
> > Looks like a problem with the blosxom renderer.  Templates use $ for
> > variables.  It allows you to escape the $ in a template with a \ so
> > that you can have dollar signs show up in templates.
> >
> > However, it looks like render_template totally does this wrong by
> > changing \$ to $ after the variables have been expanded and
> > affecting the variable expansion values.  Thus any \$ get converted
> > to $ in the story body.
> >
> > The code in the blosxom renderer that handles variable finding and
> > substitution is pretty complex.  Seems like it might be a good idea
> > to ditch that code, switch to using a real parser, and then we'd be
> > able to trivially fix this issue, too.  We'd also be able to add
> > some other features like if blocks.
> >
> > I don't think that's something I want to do in 1.5, though.  This is
> > definitely a 1.6 issue.
>
> fair enough, once we have the new bugtracker in place we can start
> gathering requirements to base the decision of the new template engine
>  
> > In the meantime, you could add a second \ or patch your copy of the
> > renderer locally.
>
> thanks, i attempted patching, but failed:
>
> diff --git a/Pyblosxom/tools.py b/Pyblosxom/tools.py
> index c560bef..f24b510 100644
> --- a/Pyblosxom/tools.py
> +++ b/Pyblosxom/tools.py
> @@ -482,6 +482,10 @@ def parse(request, var_dict, template):
>      :returns: the template string with template variables expanded.
>      """
>      encoding = request.config.get("blog_encoding", "utf-8")
> +    # after the expansion, \$ in templates will be rendered as $,
> but this will get applied
> +    # to the entire output.  To protect expanded content from this,
> we double escape first
> +    if 'body' in var_dict.keys():
> +        var_dict['body'] = var_dict['body'].replace(r'\$', '\\\$')
>      replacer = Replacer(request, encoding, var_dict)
>      return _VAR_REGEXP.sub(replacer.replace, template)
>  
>
> this kind of works, except that for some reason i need 3 backslashes
> in the replacement, and this will also change '$' into '\$', which i
> don't want, obviously. changing the first param to '\\$' doesn't help
> either.  I'm not sure what the 'r' is for, but I saw it in
> BlosxomRenderer:render_template(), and using it doesn't seem to
> change anything.
>
> Dieter

anyone an idea?  on a python commandline shell I can just do:
str.replace('\$', '\\$')
but I have no idea why I can't get it to work in Pyblosxom/tools.py

Dieter

------------------------------------------------------------------------------
Gaining the trust of online customers is vital for the success of any company
that requires sensitive data to be transmitted over the Web.   Learn how to
best implement a security strategy that keeps consumers' information secure
and instills the confidence they need to proceed with transactions.
http://p.sf.net/sfu/oracle-sfdevnl 
_______________________________________________
pyblosxom-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/pyblosxom-users
Reply | Threaded
Open this post in threaded view
|

Re: backslashes get stripped from $ signs

will kahn-greene
On 01/08/2011 04:56 PM, Dieter Plaetinck wrote:

> On Mon, 27 Dec 2010 11:46:48 +0100
> Dieter Plaetinck <[hidden email]> wrote:
>> +        var_dict['body'] = var_dict['body'].replace(r'\$', '\\\$')
>>      replacer = Replacer(request, encoding, var_dict)
>>      return _VAR_REGEXP.sub(replacer.replace, template)
>>  
>>
>> this kind of works, except that for some reason i need 3 backslashes
>> in the replacement, and this will also change '$' into '\$', which i
>> don't want, obviously. changing the first param to '\\$' doesn't help
>> either.  I'm not sure what the 'r' is for, but I saw it in
>> BlosxomRenderer:render_template(), and using it doesn't seem to
>> change anything.
>
> anyone an idea?  on a python commandline shell I can just do:
> str.replace('\$', '\\$')
> but I have no idea why I can't get it to work in Pyblosxom/tools.py

The line you're doing on the python repl is different than the one
you're doing in your patch.  The r in r'...' is for raw.  It means that
the enclosed string won't have escape sequences interpreted.  See:

http://docs.python.org/reference/lexical_analysis.html#string-literals

That's from the 2.7 docs, but it's been true in Python for a long time.

/will

------------------------------------------------------------------------------
Gaining the trust of online customers is vital for the success of any company
that requires sensitive data to be transmitted over the Web.   Learn how to
best implement a security strategy that keeps consumers' information secure
and instills the confidence they need to proceed with transactions.
http://p.sf.net/sfu/oracle-sfdevnl 
_______________________________________________
pyblosxom-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/pyblosxom-users
Reply | Threaded
Open this post in threaded view
|

Re: backslashes get stripped from $ signs

Dieter Plaetinck
On Sat, 08 Jan 2011 17:26:43 -0500
will kahn-greene <[hidden email]> wrote:

> On 01/08/2011 04:56 PM, Dieter Plaetinck wrote:
> > On Mon, 27 Dec 2010 11:46:48 +0100
> > Dieter Plaetinck <[hidden email]> wrote:
> >> +        var_dict['body'] = var_dict['body'].replace(r'\$', '\\\$')
> >>      replacer = Replacer(request, encoding, var_dict)
> >>      return _VAR_REGEXP.sub(replacer.replace, template)
> >>  
> >>
> >> this kind of works, except that for some reason i need 3
> >> backslashes in the replacement, and this will also change '$' into
> >> '\$', which i don't want, obviously. changing the first param to
> >> '\\$' doesn't help either.  I'm not sure what the 'r' is for, but
> >> I saw it in BlosxomRenderer:render_template(), and using it
> >> doesn't seem to change anything.
> >
> > anyone an idea?  on a python commandline shell I can just do:
> > str.replace('\$', '\\$')
> > but I have no idea why I can't get it to work in Pyblosxom/tools.py
>
> The line you're doing on the python repl is different than the one
> you're doing in your patch.

I know... I can't get it to work with only two slashes, like I said
below the patch. (with two slashes it doesn't change anything, maybe
I'm just looking at the wrong part of the code)

Dieter

------------------------------------------------------------------------------
Gaining the trust of online customers is vital for the success of any company
that requires sensitive data to be transmitted over the Web.   Learn how to
best implement a security strategy that keeps consumers' information secure
and instills the confidence they need to proceed with transactions.
http://p.sf.net/sfu/oracle-sfdevnl 
_______________________________________________
pyblosxom-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/pyblosxom-users
Reply | Threaded
Open this post in threaded view
|

Re: backslashes get stripped from $ signs

will kahn-greene
I talked with Dieter about this on IRC and we worked out a fix which was
just pushed.


On 01/09/2011 06:36 AM, Dieter Plaetinck wrote:

> On Sat, 08 Jan 2011 17:26:43 -0500
> will kahn-greene <[hidden email]> wrote:
>
>> On 01/08/2011 04:56 PM, Dieter Plaetinck wrote:
>>> On Mon, 27 Dec 2010 11:46:48 +0100
>>> Dieter Plaetinck <[hidden email]> wrote:
>>>> +        var_dict['body'] = var_dict['body'].replace(r'\$', '\\\$')
>>>>      replacer = Replacer(request, encoding, var_dict)
>>>>      return _VAR_REGEXP.sub(replacer.replace, template)
>>>>  
>>>>
>>>> this kind of works, except that for some reason i need 3
>>>> backslashes in the replacement, and this will also change '$' into
>>>> '\$', which i don't want, obviously. changing the first param to
>>>> '\\$' doesn't help either.  I'm not sure what the 'r' is for, but
>>>> I saw it in BlosxomRenderer:render_template(), and using it
>>>> doesn't seem to change anything.
>>>
>>> anyone an idea?  on a python commandline shell I can just do:
>>> str.replace('\$', '\\$')
>>> but I have no idea why I can't get it to work in Pyblosxom/tools.py
>>
>> The line you're doing on the python repl is different than the one
>> you're doing in your patch.
>
> I know... I can't get it to work with only two slashes, like I said
> below the patch. (with two slashes it doesn't change anything, maybe
> I'm just looking at the wrong part of the code)
>
> Dieter

------------------------------------------------------------------------------
Gaining the trust of online customers is vital for the success of any company
that requires sensitive data to be transmitted over the Web.   Learn how to
best implement a security strategy that keeps consumers' information secure
and instills the confidence they need to proceed with transactions.
http://p.sf.net/sfu/oracle-sfdevnl 
_______________________________________________
pyblosxom-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/pyblosxom-users