One Antlr grammar

I broke out the parser and the lexer because it was a pain to lex the xml and embedded doc types at the same time.  Only issue now is UTF-16.  Which may or may not be an antlr short coming.  Last post on Antlr.  Next time I'll rant on the iPhone and Android.

Here's the simplified parser and lexer.  And it really is much much simpler.  I'm thinking that if I really want to validate the XML, I'll add second pass parsing on an embedded doctype.  And entity expansion.  But for now this parses everything I need.  And the combined lexer/parser comes in at 100K vs. 2MB for Xerces.

I'm sure I can speed up performance tweaking k=x, backtrack, and memoize.  But its fine for now.

grammar SXML ;

options{ k=4; backtrack=true; memoize=true; }

/*
    this version has issues with quotes in element data
   
    starting an effort to move more logic down to the lexer
   
    and adding perserve whitespace lexwing / parsing
*/
import XMLLexer ;

// 1
xmlDocument
    : prolog element misc*
    ;

   
prolog
    : misc* doctypedecl?  misc*
    ;   
   
// [27]
misc
    : textType
    | comment
    | pi
    | WS
    ;
   
textType
    : ( UNICODE_MISC )+ { System.out.println( "found textType" ); }
    ;       

// 2.5 Comments
// [15] - should a comment be a parser rule?
comment
    : COMMENT
    ;
   
   
// 2.8   
// Document Type Definition
// [28]
doctypedecl : DOCTYPE ;


// 2.6 Processing Instructions
// [16]
pi
    : PI
    ;
   

// 3 .0
// Element
// [39]
element : ELE_EMPTY
        | ELE_OPEN content ELE_CLOSE
        ;
       
cdata
    :    CDATA ;       


// Content of Elements
// [43]
content   
    : contentItem*
    ;
   
contentItem
    : element
    | cdata 
    | pi
    | comment
    | WS
    | String
    | singleChar
    | COMMENT_CLOSE
    | PI_CLOSE
    | CDATA_CLOSE   
    ;   
   
singleChar
    : BANG
    | DQUOTE
    | POUND
    | DOLLAR
    | PCT
    | AMP
    | SQUOTE
    | LPAREN
    | RPAREN
    | STAR
    | PLUS
    | COMMA
    | MINUS
    | MINUSMINUS
    | DOT
    | FWD_SLASH
    | DIGIT
    | COLON
    | SEMI
    | LT
    | EQ
    | GT
    | QUEST
    | AT
    | UPPER
    | LBRACKET
    | BACK_SLASH
    | RBRACKET
    | CARET
    | UNDER_SCORE
    | BACK_TICK
    | LOWER
    | PIPE
    | TILDE
    | LCURLY
    | RCURLY
    | UNICODE_MISC
    | SINK
    ;   

lexer grammar XMLLexer ;

options{ k=4; backtrack=true; memoize=true; }

/*
    A lexer to create tokens for the xml grammar
*/


PI : PI_OPEN ( options{greedy=false;}: . )* PI_CLOSE ;
PI_OPEN : 'PI_CLOSE : '?>' ;

COMMENT : COMMENT_OPEN ( options{greedy=false;}: . )* COMMENT_CLOSE ;
COMMENT_OPEN : '' ;

CDATA : CDATA_OPEN ( options{greedy=false;} : . )*  CDATA_CLOSE ;
fragment CDATA_OPEN : '' ;

INCLUDE : INCLUDE_OPEN ( options{greedy=false;} : . )*  CDATA_CLOSE ;
INCLUDE_OPEN : '
CARET : '^' ;   
   
// 005F   
UNDER_SCORE : '_' ;   

BACK_TICK :    '`' ;

//     0061..007A
LOWER : 'a'..'z' ;

// 007D
PIPE : '|' ;

// 007E
TILDE : '~' ;

LCURLY : '{' ;
RCURLY : '}' ;

UNICODE_MISC
    : '\u00A0'..'\uD7FF'
    | '\uE000'..'\uFFFD' ;
   
// [3]
// #x20 ' ' \u0020
// #xA '\n'  \u000A
// #xD '\r'  \u000D
// #x9 '\t' \u0009
// do not preserve whitespace {$channel=HIDDEN;}
WS
    : ( '\r' | '\n' | '\t' | ' ' )+
    ;
   
SINK
    : . { System.out.println("found sink"); };