[metaas-dev] No viable alternative error

[David Holroyd]

On Wed, Mar 07, 2007 at 04:48:16AM +0100, Martin Schnabel wrote:
> David Holroyd schrieb:
> >Adding these to the AS3 grammar brings a noticeable slowdown to the unit
> >tests, probably due to the fact that metaas is constantly creating
> >parser and lexer instances on the fly, and lexer initialisation is now
> >vastly more expensive.  Adding these rules makes the generated .java
> >file grow by almost 5000 lines.  :(

> since in most cases no special chars are needed and they would only 
> change the lexer why not use a lexer factory and support customized 
> lexer that support different kinds of languages. so there is one lexer 
> for greek one for latin languages. another for korean etc.
> just an idea. not a good one but... better then handwriting a custom lexer.
> or just exclude kanjis and other pictogram chars like that. they have 
> quite a big range.
> it seams that in as2 no unicode is allowed in identifiers. however mtasc 
> does not support them so i dont need to implement it in asdt.

My hope is that all the additional cost is paid in the call,

  new AS3Lexer(..)

and that by caching the lexer instance and reusing it, things will
perform just as they did before, even with all of the aditional
characters which are supported.  (The lexer in not thread-safe, so there
can't just be a singleton instance shared by the whole app.)

I've yet to investigate properly though.


> would be interesting which range is actually supported by the flex sdk, 
> or did you found a better specification?

Based on our earlier discussion of 'unicode identifiers' I wrote the
attached program to generate the lexer rules.  Its quite simple, just
calls to isUnicodeIdentifierStart() and isUnicodeIdentifierPart(), plus
some work to build the formatted output...



-- 
http://david.holroyd.me.uk/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: UnicodeIdentifierGenerator.java
Type: text/x-java
Size: 2309 bytes
Desc: not available
Url : http://lists.badgers-in-foil.co.uk/pipermail/metaas-dev/attachments/20070307/abfef6e7/UnicodeIdentifierGenerator.java