Some insight into javaCC from a friend of jython

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Some insight into javaCC from a friend of jython

Walter Chang-2
Hi All,

I don't know about you, but parser, lexers, compilier's compilers have
always seem to be a bit difficult to understand.

I took the liberty of approaching Fabio from the excellent pydev
eclipse plugin project about some thoughts on how the javaCC part of
jython works, which is a very important part to understand to get a
good big picture of jython development (i think...)

It would be great for Fank and Samuele to add to this dialogue.  It
would be a great intro into the innards of jython for someone wanting
to contribute to read as a primer.

I think if we ask nicely, Fabio would be willing to help out where he
can...(he's a bit tied up rigth now though...)

Walter:

Based on the work that you did on jython integration into pydev, can
you give us some pointers as to how the javaCC part of it works ?

What would be necessary to build upon the work that you've done and
integrate it into jython? (daunting task...)

Fabio:

Ok, I'm not sure on how to complete the whole integration, as I did not
get too much into jython itself, just its grammar, but it works
something as:

1. You create a 'tree' definition that you use to specify the jython
language in a 'logic' way (that's the Python.asdl file)
2. You use the script asdl_java.py to create Java files that represent
that structure (they use the visitor pattern later) -- this is the AST
(abstract syntax tree).
3. You create the javaCC definition (that's python.jjt). That definition
will parse the jython code and create the AST at runtime.
4. The AST is used as the logic structure inside jython, and interpreted
accordingly (This is the part I'm not familiar with)... I do many things
in pydev with that AST, from getting information to do code-completion,
getting the outline of the code and even making code-analysis... so,
while I do use that structure, I'm not sure how jython uses it.

So, I could help you getting the grammar up to the last version (pydev
already has it)... but right now, it's not a matter of just 'get it', as
the grammar evolved much more than it would be needed for jython (it
gathers more info about things such as comments and other tokens such as
'(' or '[' for making pretty-printing, which you just don't need at
jython, the exact position for class / function names for code analysis,
etc). So, what should be done is: get the Python.asdl from the CPython
project -- as far as I know, it should be the same structure (as I said,
in pydev you have much more things that will just distract you in the
end), regenerate the AST, then make the javaCC parse the code for the
new constructs (you can use the pydev grammar as a reference for that)
and change the TreeBuilder.java (that's the class that actually builds
the AST at each new construct found in javaCC) to build those new
constructs correctly... (I guess you could use the TreeBuilder in pydev
as a reference too).

--
Simplicity--the art of maximizing the amount
of work not done--is essential.
from: http://www.agilemanifesto.org/principles.html


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev
Reply | Threaded
Open this post in threaded view
|

Re: Some insight into javaCC from a friend of jython

Walter Chang-2
test

On 3/30/06, Walter Chang <[hidden email]> wrote:

> Hi All,
>
> I don't know about you, but parser, lexers, compilier's compilers have
> always seem to be a bit difficult to understand.
>
> I took the liberty of approaching Fabio from the excellent pydev
> eclipse plugin project about some thoughts on how the javaCC part of
> jython works, which is a very important part to understand to get a
> good big picture of jython development (i think...)
>
> It would be great for Fank and Samuele to add to this dialogue.  It
> would be a great intro into the innards of jython for someone wanting
> to contribute to read as a primer.
>
> I think if we ask nicely, Fabio would be willing to help out where he
> can...(he's a bit tied up rigth now though...)
>
> Walter:
>
> Based on the work that you did on jython integration into pydev, can
> you give us some pointers as to how the javaCC part of it works ?
>
> What would be necessary to build upon the work that you've done and
> integrate it into jython? (daunting task...)
>
> Fabio:
>
> Ok, I'm not sure on how to complete the whole integration, as I did not
> get too much into jython itself, just its grammar, but it works
> something as:
>
> 1. You create a 'tree' definition that you use to specify the jython
> language in a 'logic' way (that's the Python.asdl file)
> 2. You use the script asdl_java.py to create Java files that represent
> that structure (they use the visitor pattern later) -- this is the AST
> (abstract syntax tree).
> 3. You create the javaCC definition (that's python.jjt). That definition
> will parse the jython code and create the AST at runtime.
> 4. The AST is used as the logic structure inside jython, and interpreted
> accordingly (This is the part I'm not familiar with)... I do many things
> in pydev with that AST, from getting information to do code-completion,
> getting the outline of the code and even making code-analysis... so,
> while I do use that structure, I'm not sure how jython uses it.
>
> So, I could help you getting the grammar up to the last version (pydev
> already has it)... but right now, it's not a matter of just 'get it', as
> the grammar evolved much more than it would be needed for jython (it
> gathers more info about things such as comments and other tokens such as
> '(' or '[' for making pretty-printing, which you just don't need at
> jython, the exact position for class / function names for code analysis,
> etc). So, what should be done is: get the Python.asdl from the CPython
> project -- as far as I know, it should be the same structure (as I said,
> in pydev you have much more things that will just distract you in the
> end), regenerate the AST, then make the javaCC parse the code for the
> new constructs (you can use the pydev grammar as a reference for that)
> and change the TreeBuilder.java (that's the class that actually builds
> the AST at each new construct found in javaCC) to build those new
> constructs correctly... (I guess you could use the TreeBuilder in pydev
> as a reference too).
>
> --
> Simplicity--the art of maximizing the amount
> of work not done--is essential.
> from: http://www.agilemanifesto.org/principles.html
>


--
Simplicity--the art of maximizing the amount
of work not done--is essential.
from: http://www.agilemanifesto.org/principles.html


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid0944&bid$1720&dat1642
_______________________________________________
Jython-dev mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jython-dev