Getting back to the DSL

[Tarek Ziadé]

Hey

I am happy with my DSL, I can yacc.parse() it it and create an AST in memory.

I am now trying to add a feature where, for a given AST, I can
generate the corresponding portion of DSL.

The AST I've created in memory is a simple tuple for every expression
I am visiting, and I was wondering what is the best way to do the
revert operation.

The problem I am facing is that I don't want to maintain on one side a
grammar to parse my DSL, and on the other side a function to parse the
AST and do the revert operation.

I was wondering how this is usually solved, or if PLY provides some
helpers for this

Thanks
Tarek

[Michael Foord]

Post by Tarek Ziadé
Hey
I am happy with my DSL, I can yacc.parse() it it and create an AST in memory.
I am now trying to add a feature where, for a given AST, I can
generate the corresponding portion of DSL.
The AST I've created in memory is a simple tuple for every expression
I am visiting, and I was wondering what is the best way to do the
revert operation.
The problem I am facing is that I don't want to maintain on one side a
grammar to parse my DSL, and on the other side a function to parse the
AST and do the revert operation.
I was wondering how this is usually solved, or if PLY provides some
helpers for this

If the AST contains (stores) the original tokens the collapsing an AST back
to "code" should be trivial. At resolver systems we used PLY to rewrite a
custom language to Python via AST transformations. Each AST node had a
"collapse" method (recursive) that would return a string representing the
node.

Having parsed you could get code back by calling collapse on the top level
node.

All the best,

Michael Foord

Post by Tarek Ziadé
Thanks
Tarek
--
Tarek Ziadé | http://ziade.org
--
You received this message because you are subscribed to the Google Groups "ply-hack" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/ply-hack?hl=en.

--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

[Tarek Ziadé]

On Tue, Sep 27, 2011 at 1:29 PM, Michael Foord <***@gmail.com> wrote:
..

Post by Michael Foord
If the AST contains (stores) the original tokens the collapsing an AST back
to "code" should be trivial. At resolver systems we used PLY to rewrite a
custom language to Python via AST transformations. Each AST node had a
"collapse" method (recursive) that would return a string representing the
node.
Having parsed you could get code back by calling collapse on the top level
node.

That sounds like the best idea. I'll try this.

Do you happen to have a public example ?

Cheers

Post by Michael Foord
Michael Foord

Post by Tarek Ziadé
Thanks
Tarek
--
Tarek Ziadé | http://ziade.org
--
You received this message because you are subscribed to the Google Groups
"ply-hack" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/ply-hack?hl=en.

--
http://www.voidspace.org.uk/
May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
--
You received this message because you are subscribed to the Google Groups "ply-hack" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/ply-hack?hl=en.

[David Beazley]

I think you just do a depth-first traversal of the parse tree. For example something sort of like this (shown for a node representing a binary operator):

class BinaryOperator(object):
def __init__(self, operator, left, right):
self.operator = operator
self.left = left
self.right = right
def collapse(self):
return self.left.collapse() + self.operator + self.right.collapse()

Note: You'll need to careful about too much recursion depending on the depth of your parse tree.

Cheers,
Dave

Post by Tarek Ziadé
..

Post by Michael Foord
If the AST contains (stores) the original tokens the collapsing an AST back
to "code" should be trivial. At resolver systems we used PLY to rewrite a
custom language to Python via AST transformations. Each AST node had a
"collapse" method (recursive) that would return a string representing the
node.
Having parsed you could get code back by calling collapse on the top level
node.

That sounds like the best idea. I'll try this.
Do you happen to have a public example ?
Cheers

Post by Michael Foord
Michael Foord

Post by Tarek Ziadé
Thanks
Tarek
--
Tarek Ziadé | http://ziade.org
--
You received this message because you are subscribed to the Google Groups
"ply-hack" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/ply-hack?hl=en.

--
http://www.voidspace.org.uk/
May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
--
You received this message because you are subscribed to the Google Groups
"ply-hack" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/ply-hack?hl=en.

--
Tarek Ziadé | http://ziade.org
--
You received this message because you are subscribed to the Google Groups "ply-hack" group.
For more options, visit this group at http://groups.google.com/group/ply-hack?hl=en.

[Tarek Ziadé]

             self.operator = operator
             self.left = left
             self.right  = right
             return self.left.collapse() + self.operator + self.right.collapse()
Note:  You'll need to careful about too much recursion depending on the depth of your parse tree.

Thanks, that's useful

Cheers
Tarek

[Michael Foord]

Post by David Beazley

Post by David Beazley
I think you just do a depth-first traversal of the parse tree. For

example something sort of like this (shown for a node representing a binary

Post by David Beazley
self.operator = operator
self.left = left
self.right = right
return self.left.collapse() + self.operator +

self.right.collapse()

Post by David Beazley
Note: You'll need to careful about too much recursion depending on the

depth of your parse tree.
Thanks, that's useful

You may want your grammar to preserve whitespace in the tokens (rather than
discarding) - unless you're happy to lose / change whitespace when round
tripping.

Michael

Post by David Beazley
Cheers
Tarek
--
Tarek Ziadé | http://ziade.org
--
You received this message because you are subscribed to the Google Groups "ply-hack" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/ply-hack?hl=en.

--
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html

[Tarek Ziadé]

On Tue, Sep 27, 2011 at 2:49 PM, Michael Foord <***@gmail.com> wrote:
...

Post by Michael Foord
You may want your grammar to preserve whitespace in the tokens (rather than
discarding) - unless you're happy to lose / change whitespace when round
tripping.

Yeah ideally I'd like the round trip to look similar, including the
level indentation.

The use case is a web editor where you can change pieces of an already
loaded AST, and have the DSL rendered back and forth.

If the first sync changes the initial text after the first
tokenization, it's fine I guess.

Post by Michael Foord
Michael

Post by Tarek Ziadé
Cheers
Tarek
--
Tarek Ziadé | http://ziade.org
--
You received this message because you are subscribed to the Google Groups
"ply-hack" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/ply-hack?hl=en.

--
http://www.voidspace.org.uk/
May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
--
You received this message because you are subscribed to the Google Groups "ply-hack" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/ply-hack?hl=en.