From klangner at gmail.com Thu Apr 9 09:03:45 2009 From: klangner at gmail.com (Krzysztof Langner) Date: Thu, 9 Apr 2009 10:03:45 +0200 Subject: [metaas-dev] compiling grammar with ANTLR 3.1.3 Message-ID: Hi, First thank you for great job on metaas. This library has very nice api and is well thought. I want to make a program which will analyze AS code and find suspicious places (like public static not final class members). Since I couldn't find AS parser in flex SDK it looks that metaas is natural choice for this project. Unfortunately there is one big problem. Metaas requires that each instruction is ended by semicolon. And in AS you can omit them. So most of the files which I want to analyze just want compile. So I decided to look at the code and try to make necessary changes to the grammar. But first I wanted to compile this program with ant and add testcase with the problem. I already have build.xml but there is a problem with the AS3.g3 grammar. So I have some questions regarding this file: Is there a reason that this file got g3 extension instead of g? The problem is that ANTLR3 doesn't like this extension and don't generate any output. When I change extenstion to .g I got this error message: ANTLR could not analyze this decision in rule Tokens; often this is because of recursive rule references visible from the left edge of alternatives. ANTLR will re-analyze the decision with a fixed lookahead of k=1. Consider using "options {k=1;}" for that decision and possibly adding a syntactic predicate. And then ANTLR crashes: internal error: org.antlr.tool.Grammar.createLookaheadDFA(Grammar.java:1279): could not even do k=1 for decision 26; reason: timed out (>1000ms) Which version of ANTLR should I use to compile this grammar? Best regards Krzysztof Langner -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.badgers-in-foil.co.uk/pipermail/metaas-dev/attachments/20090409/ee6b31cc/attachment.htm From dave at badgers-in-foil.co.uk Thu Apr 9 09:43:36 2009 From: dave at badgers-in-foil.co.uk (David Holroyd) Date: Thu, 9 Apr 2009 09:43:36 +0100 Subject: [metaas-dev] compiling grammar with ANTLR 3.1.3 In-Reply-To: References: Message-ID: <20090409084336.GA26722@badgers-in-foil.co.uk> On Thu, Apr 09, 2009 at 10:03:45AM +0200, Krzysztof Langner wrote: > Hi, > > First thank you for great job on metaas. This library has very nice api and > is well thought. > > I want to make a program which will analyze AS code and find suspicious > places (like public static not final class members). > Since I couldn't find AS parser in flex SDK it looks that metaas is natural > choice for this project. > > Unfortunately there is one big problem. Metaas requires that each > instruction is ended by semicolon. And in AS you can omit them. > So most of the files which I want to analyze just want compile. > > So I decided to look at the code and try to make necessary changes to the > grammar. But first I wanted to compile this program with ant and add > testcase with the problem. > > I already have build.xml but there is a problem with the AS3.g3 grammar. > So I have some questions regarding this file: > > Is there a reason that this file got g3 extension instead of g? > The problem is that ANTLR3 doesn't like this extension and don't generate > any output. > > When I change extenstion to .g I got this error message: > > ANTLR could not analyze this decision in rule Tokens; often this is because > of recursive rule references visible from the left edge of alternatives. > ANTLR will re-analyze the decision with a fixed lookahead of k=1. Consider > using "options {k=1;}" for that decision and possibly adding a syntactic > predicate. > > And then ANTLR crashes: > > internal error: > org.antlr.tool.Grammar.createLookaheadDFA(Grammar.java:1279): could not even > do k=1 for decision 26; reason: timed out (>1000ms) > > > Which version of ANTLR should I use to compile this grammar? Have you tried the Maven build included with the source? svn co http://svn.badgers-in-foil.co.uk/metaas/trunk/ metaas cd metaas mvn package Admittedly, I just tried this with a fresh checkout to check the process is working, and got a unit test failure, SourceFolderResourceRootTest: expected: but was: Oops! However, by the time the unit tests are run, the grammar processing has taken place, the generated Java code has been compiled etc. Does this help? -- http://david.holroyd.me.uk/ From dave at badgers-in-foil.co.uk Thu Apr 9 09:52:43 2009 From: dave at badgers-in-foil.co.uk (David Holroyd) Date: Thu, 9 Apr 2009 09:52:43 +0100 Subject: [metaas-dev] compiling grammar with ANTLR 3.1.3 In-Reply-To: References: Message-ID: <20090409085243.GB26722@badgers-in-foil.co.uk> On Thu, Apr 09, 2009 at 10:03:45AM +0200, Krzysztof Langner wrote: > Unfortunately there is one big problem. Metaas requires that each > instruction is ended by semicolon. And in AS you can omit them. > So most of the files which I want to analyze just want compile. Oh, I should also warn you that the semicolon handling is not the only point at which the grammar in metaas is incomplete. If you are going to be processing large numbers of arbitrary AS3 files, you've got to expect to run up against examples of other syntax which I did not get around to handling. It's been a while since I was hacking on the grammar, but IIRC there are also assumptions about what constructs are allowed to appear in package/class bodies (i.e. metaas is more restrictive than the spec), and probably other things that I don't even know about. ta, dave -- http://david.holroyd.me.uk/ From klangner at gmail.com Thu Apr 9 10:58:07 2009 From: klangner at gmail.com (Krzysztof Langner) Date: Thu, 9 Apr 2009 11:58:07 +0200 Subject: [metaas-dev] compiling grammar with ANTLR 3.1.3 In-Reply-To: <20090409084336.GA26722@badgers-in-foil.co.uk> References: <20090409084336.GA26722@badgers-in-foil.co.uk> Message-ID: Thank you for fast answer :-). I haven't tried maven build yet. But since it looks that I'll have to work on this code to make it work for my project I need to be able to compile it with ant and get it working under Eclipse. I work with the source from SVN and couldn't find ANTLR libs there so I just downloaded lastest version 3.1.3 from antlr website and got some problems. It looks that there are some changes between antlr 3.0 and 3.1 which cause my problems. Anyway I understand that you don't have currently time to work on this library (metaas). I think I'll try to make it work with ant and eclipse with antlr 3.1 and then I'll try to fix unit tests. If I make it work I'll post modified sources with build.xml (if you are interested). But can you can answer 2 questions: Is there any reason that AS3 grammar has extension .g3 instead of .g? Do I need any command line switch for antlr to parse AS3.g3 grammar? Have you tried the Maven build included with the source? > > svn co http://svn.badgers-in-foil.co.uk/metaas/trunk/ metaas > cd metaas > mvn package > > Admittedly, I just tried this with a fresh checkout to check the > process is working, and got a unit test failure, > > SourceFolderResourceRootTest: expected: but was: > > Oops! However, by the time the unit tests are run, the grammar > processing has taken place, the generated Java code has been compiled > etc. > > > Does this help? > > -- > http://david.holroyd.me.uk/ > > _______________________________________________ > metaas-dev mailing list > metaas-dev at lists.badgers-in-foil.co.uk > http://lists.badgers-in-foil.co.uk/mailman/listinfo/metaas-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.badgers-in-foil.co.uk/pipermail/metaas-dev/attachments/20090409/fab42bc7/attachment.htm From dave at badgers-in-foil.co.uk Thu Apr 9 11:15:17 2009 From: dave at badgers-in-foil.co.uk (David Holroyd) Date: Thu, 9 Apr 2009 11:15:17 +0100 Subject: [metaas-dev] compiling grammar with ANTLR 3.1.3 In-Reply-To: References: <20090409084336.GA26722@badgers-in-foil.co.uk> Message-ID: <20090409101517.GA28659@badgers-in-foil.co.uk> On Thu, Apr 09, 2009 at 11:58:07AM +0200, Krzysztof Langner wrote: > Thank you for fast answer :-). > I haven't tried maven build yet. But since it looks that I'll have to work > on this code to make it work for my project I need to be able to compile it > with ant and get it working under Eclipse. I work with the source from SVN > and couldn't find ANTLR libs there so I just downloaded lastest version > 3.1.3 from antlr website and got some problems. Note that mvn eclipse:eclipse will generate .project and .classpath files so that you can 'Import exiating project into workspace' in Eclipse. Also, mvn ant:ant will generate a build.xml. It doesn't understand how to translate the ANTLR plugin usage, but using this approach would at least allow stuff like the list of dependencies to be managed in a single place. > It looks that there are some changes between antlr 3.0 and 3.1 which cause > my problems. I've not yet used 3.1, so I can't be sure. > Anyway I understand that you don't have currently time to work on this > library (metaas). > I think I'll try to make it work with ant and eclipse with antlr 3.1 and > then I'll try to fix unit tests. > If I make it work I'll post modified sources with build.xml (if you are > interested). I'd be very happy to include contributed patches! (Best of all with unit tests.) If you keep the folder structure unchanged then I will be able to continue using Maven to build, and > But can you can answer 2 questions: > Is there any reason that AS3 grammar has extension .g3 instead of .g? No. It is simply named this way because I originally started with an ANTLR2 grammar (filename ended with .g), and kept both versions side-by-side for a while. I will rename the file now to make your life easier. > Do I need any command line switch for antlr to parse AS3.g3 grammar? I don't think so. You can see in the antlr3-maven-plugin section of pom.xml that the only thing of note is that 'Regex.g' is excluded from the build. You might need to add some switches to manage the inter-grammar dependencies too (the maven-antlr3-plugin does that automatically). -- http://david.holroyd.me.uk/ From velo.br at gmail.com Tue Apr 21 18:05:25 2009 From: velo.br at gmail.com (Marvin Froeder) Date: Tue, 21 Apr 2009 14:05:25 -0300 Subject: [metaas-dev] Support for namespace and function classes Message-ID: Hi there, Is there any plans for supporting namespace classes: acme_internal.as package com.acme { public namespace acme_internal = "http://www. acme.com/internal"; } And how about function classes: JustFunction.as package com.acme { public function JustFunction():Object { return new Object(); } } VELO -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.badgers-in-foil.co.uk/pipermail/metaas-dev/attachments/20090421/5f388941/attachment.htm From velo.br at gmail.com Tue Apr 21 18:09:08 2009 From: velo.br at gmail.com (Marvin Froeder) Date: Tue, 21 Apr 2009 14:09:08 -0300 Subject: [metaas-dev] Support for namespace and function classes In-Reply-To: References: Message-ID: Hi there, Is there any plans for supporting namespace classes: acme_internal.as package com.acme { public namespace acme_internal = "http://www. acme.com/internal"; } And how about function classes: JustFunction.as package com.acme { public function JustFunction():Object { return new Object(); } } VELO -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.badgers-in-foil.co.uk/pipermail/metaas-dev/attachments/20090421/ab732a70/attachment.htm