Tuesday, September 22, 2009

Antlr XML Grammar

Fixed an issue with quotes inside a comment or cdata section

grammar XML ;

options {
    backtrack=true;
}
// 1
xmlDocument
    : prolog element misc*
    ;
   
prolog
    : xmlDecl misc* (doctypedecl  misc*)?
    ;
   
// [27]
misc
    : comment
    | pi
    ;    

// 2.5 Comments
// [15] - should a comment be a parser rule?
/*
comment
    : CommentOpen commentContent* CommentClose
    ;
   
commentContent
    : ~CommentClose
    ;   
*/
   
comment
    : COMMENT
    ;
   
COMMENT
    : CommentOpen ( options{greedy=false;}: . )* CommentClose   
    ;
   
// 2.8   
// Document Type Definition
// [28]
doctypedecl : DocTypeOpen Name (externalID)? ( LBRACKET intSubset RBRACKET )? GT ;


// [28b]      

intSubset : (markupdecl | declSep)* ;

// [29]
markupdecl
    : elementdecl
    | attlistDecl
    | entityDecl
    | notationDecl
    | pi
    | comment    
    ;
   
// 2.6 Processing Instructions
// [16]
pi
    : PIOpen PITarget PIData* PIClose
    ;
   
// External Subset
// [30]
extSubset : textDecl? extSubsetDecl ;

// [31]
extSubsetDecl : ( markupdecl | conditionalSect | declSep)*     ;


// 3 .0
// Element
// [39]
element : emptyElemTag
        | sTag content eTag
        ;


// Content of Elements
// [43]
content   
    : contentItem*
    ;
   
contentItem
    : element
    | reference
    | CDSect
    | pi
    | comment
    | charData
    ;   
   
// 2.4 Character Data and Markup
// [14]
charData
    : charDataItem+
    ;


charDataItem
    : ~( LT
    | AMP
    | CDataEnd
    | CDataStart
    | CommentOpen
    | ETagOpen
    | ElementDeclOpen
    | AttListDeclOpen
    | EntityRef
    | NotationDeclOpen
    | PIOpen
    | STagOpen )
    ;   
   

// 3.2
// Element Type Declaration
// [45]
elementdecl : ElementDeclOpen Name contentspec GT ;



// [46]
contentspec
    : Empty
    | Any
    | mixed
    | children
    ;

// 3.2.1   
// Element-content Models
// [47]
children
    : ( choice | seq ) modifier?
    ;

// [48]   
cp
    : (Name | choice | seq) modifier?
    ;
   
// [49]
choice
    : LPAREN cp ( PIPE cp )+ RPAREN
    ;
   
// [50]
seq   
    : LPAREN cp ( COMMA cp )* RPAREN
    ;
   
modifier : ( QUEST | STAR | PLUS ) ;
   
   
// 3.2.2
// Mixed-content Declaration
// [51]
mixed : LPAREN PCData ( PIPE Name )*  RPAREN
    | LPAREN PCData RPAREN
    ;
   

// 3.3   
// Attribute-list Declaration
// [52]
attlistDecl : AttListDeclOpen Name (attDef)* GT ;

// [53]
attDef : Name attType defaultDecl ;
               
               
// 3.3.1
// Attribute Types
// [54]
attType : stringType | tokenizedType | enumeratedType ;

// [55]
stringType : CDataType ;



// [56]
tokenizedType
    : ID   
    | IDREF
    | IDREFS   
    | ENTITY   
    | ENTITIES   
    | NMTOKEN   
    | NMTOKENS
    ;
   
   
   
// Enumerated Attribute Types
// [57]
enumeratedType
    : notationType
    | enumeration
    ;
   
// [58]
notationType
    : NOTATION LPAREN Name ( PIPE Name)* RPAREN
    ;

enumeration
    : LPAREN (Name) ( PIPE (Name) )*  RPAREN
    ;
   
// 3.3.2
// Attribute Defaults
// [60]
defaultDecl
    : Required
    | Implied
    | (Fixed)? StringLiteral
    ;

   
                               
                               
// 3.4                               
// Conditional Section
// [61]
conditionalSect
    : includeSect
    | ignoreSect
    ;
   
// [62]
includeSect
    : Cond_Open Include LBRACKET extSubsetDecl CDataEnd
    ;
   
// [63]
ignoreSect
    : Cond_Open IGNORE LBRACKET ignoreSectContents* CDataEnd 
    ;
   
ignoreSectContents
    : IgnoreItem ( Cond_Open ignoreSectContents CDataEnd IgnoreItem)*
    ;
   


// 4.3
// Text Declaration
// [77]
textDecl : XMLDeclOpen versionInfo? encodingDecl PIClose ;

// Well-Formed External Parsed Entity
// [78]
extParsedEnt
    : textDecl? content
    ;
   
// Encoding Declaration
// [80]
encodingDecl : Encoding EQ StringLiteral ;



// 4.7
// [82]
notationDecl
    : NotationDeclOpen Name  (externalID | publicID) GT
    ;

// [83]
publicID
    : PublicToken StringLiteral
    ;
   

// 4.2.2 External Entities
// [75]
externalID
    : SystemToken StringLiteral
    | PublicToken StringLiteral StringLiteral
    ;
   
// [76]
nDataDecl
    : NData Name
    ;


// 4.2
// Entity Declaration
// [70]
entityDecl
    : EntityDeclOpen PCT? Name entityDef GT
    ;
   
// [73]
entityDef
    : StringLiteral
    | externalID nDataDecl?
    ;
   

   
   
// 4.2   
// Entity Reference
// [67]      
reference   
    : EntityRef
    | CharRef
    ;
   

// 3.1       
// Start-tag

// [40]
sTag : STagOpen (attribute|versionInfo)* GT ;



// [41]
attribute : Name EQ StringLiteral ;

// End-tag
//[42]
eTag : ETagOpen Name GT ;


// Tags for Empty Elements
// [44]
emptyElemTag : STagOpen (attribute)*  ETagClose ;


   

// 2.9
// Standalone Document Declaration
// [32]
sdDecl : Standalone EQ StringLiteral ;
   

// [28a]
declSep : PEReference ;

// [23]
xmlDecl    : XMLDeclOpen versionInfo (encodingDecl)? (sdDecl)? PIClose ;



// [24]
versionInfo : Version EQ StringLiteral ;


// LEXER



// 4.1
// Character Reference
// [66]
CharRef   
    : '&#' DIGIT+ ';'
    | '&#x' ( DIGIT | 'a'..'f' | 'A'..'F' )+ ';'   
    ;
   
   
// [68]
EntityRef : '&' Name ';' ;

// [69]
PEReference : '%' Name ';' ;   



STagOpen : LT Name ;



ETagOpen : '
   
ETagClose :    '/>' ;   


DocTypeOpen : '


XMLDeclOpen : '
Version : 'version' ;
   
// [65]
fragment IgnoreItem
    : ~( Cond_Open | CDataEnd )
    ;
   
Cond_Open : '// move to Parser?
// CDSect : CDataStart  CData*  CDataEnd ;

CDSect : CDataStart  ( options{greedy=false;} : . )*  CDataEnd ;

// [19]
CDataStart : '// - (('X' | 'x') ('M' | 'm') ('L' | 'l'))
fragment PITarget : Name  ;


PIOpen : 'PIClose : '?>' ;

   

CommentOpen : '' ;



   
// Key words and long tokens

   
CDataEnd : ']]>' ;

ElementDeclOpen : '
// 0028
fragment LPAREN : '(';

// 0029   
fragment RPAREN : ')' ;

// 002A
STAR :    '*' ;

// 002B
fragment PLUS : '+';

// 002C
fragment COMMA : ',';
   
// 002D   
MINUS : '-' ;

// 002E
DOT : '.' ;

// 002F
FWD_SLASH : '/' ;

// 0030..0039
DIGIT : '0'..'9';

// 003A   
fragment COLON : ':' ;   

// 003B
fragment SEMI : ';' ;

// 003C
fragment LT : '<' ;

// 003D
EQ : '=' ;

// 003E
GT : '>' ;   

// 003F
fragment QUEST : '?' ;

// 0040
fragment AT : '@' ;

// 0041..005A
fragment UPPER : 'A'..'Z' ;

// 005B
fragment LBRACKET : '[' ;

// 005C
fragment BACK_SLASH  :    '\\';

// 005D
fragment RBRACKET :    ']';

   
// 005F   
fragment UNDER_SCORE :    '_' ;   


//     0061..007A
fragment LOWER : 'a'..'z' ;

// 007D
fragment PIPE : '|' ;

// [3]
// #x20 ' ' \u0020
// #xA '\n'  \u000A
// #xD '\r'  \u000D
// #x9 '\t' \u0009
WS
    : ( '\r' | '\n' | '\t' | ' ' )+ {$channel=HIDDEN;}
    ;

Monday, September 21, 2009

ANTRL XML

Here is a little Antlr grammar for XML.
Antlr is a tool that I've always found interesting. And I found a use for it. I was sick and tired of the typical Java mechanism for loading xml files. Factory, Builder, Parser. Then maybe get a DOM. Then of course trying to turn off validations or white space with cryptic URL property settings which you can never find documentation on.

So, here's the grammar. Is it full proof? I don't know. Does it validate? No. It parsers xml.
Is it the best Antlr grammar? Probably not.

Does it load the web.xml, hibernate configs, struts configs, etc. that I need yes. It is fast. Oh, yeah.

Here it is. The numbering is taken from the W3C spec.



grammar XML ;

options {
backtrack=true;
}
// 1
xmlDocument
: prolog element misc*
;

prolog
: xmlDecl misc* (doctypedecl misc*)?
;

// [27]
misc
: comment
| pi
;

// 2.5 Comments
// [15] - should a comment be a parser rule?
comment
: CommentOpen commentContent CommentClose
;

commentContent
: ~CommentClose
;


// 2.8
// Document Type Definition
// [28]
doctypedecl : DocTypeOpen Name (externalID)? ( LBRACKET intSubset RBRACKET )? GT ;


// [28b]

intSubset : (markupdecl | declSep)* ;

// [29]
markupdecl
: elementdecl
| attlistDecl
| entityDecl
| notationDecl
| pi
| comment
;

// 2.6 Processing Instructions
// [16]
pi
: PIOpen PITarget PIData* PIClose
;

// External Subset
// [30]
extSubset : textDecl? extSubsetDecl ;

// [31]
extSubsetDecl : ( markupdecl | conditionalSect | declSep)* ;


// 3 .0
// Element
// [39]
element : emptyElemTag
| sTag content eTag
;


// Content of Elements
// [43]
content
: contentItem*
;

contentItem
: element
| reference
| CDSect
| pi
| comment
| charData
;

// 2.4 Character Data and Markup
// [14]
charData
: charDataItem+
;


charDataItem
: ~( LT
| AMP
| CDataEnd
| CDataStart
| CommentOpen
| ETagOpen
| ElementDeclOpen
| AttListDeclOpen
| EntityRef
| NotationDeclOpen
| PIOpen
| STagOpen )
;


// 3.2
// Element Type Declaration
// [45]
elementdecl : ElementDeclOpen Name contentspec GT ;



// [46]
contentspec
: Empty
| Any
| mixed
| children
;

// 3.2.1
// Element-content Models
// [47]
children
: ( choice | seq ) modifier?
;

// [48]
cp
: (Name | choice | seq) modifier?
;

// [49]
choice
: LPAREN cp ( PIPE cp )+ RPAREN
;

// [50]
seq
: LPAREN cp ( COMMA cp )* RPAREN
;

modifier : ( QUEST | STAR | PLUS ) ;


// 3.2.2
// Mixed-content Declaration
// [51]
mixed : LPAREN PCData ( PIPE Name )* RPAREN
| LPAREN PCData RPAREN
;


// 3.3
// Attribute-list Declaration
// [52]
attlistDecl : AttListDeclOpen Name (attDef)* GT ;

// [53]
attDef : Name attType defaultDecl ;


// 3.3.1
// Attribute Types
// [54]
attType : stringType | tokenizedType | enumeratedType ;

// [55]
stringType : CDataType ;



// [56]
tokenizedType
: ID
| IDREF
| IDREFS
| ENTITY
| ENTITIES
| NMTOKEN
| NMTOKENS
;



// Enumerated Attribute Types
// [57]
enumeratedType
: notationType
| enumeration
;

// [58]
notationType
: NOTATION LPAREN Name ( PIPE Name)* RPAREN
;

enumeration
: LPAREN (Name) ( PIPE (Name) )* RPAREN
;

// 3.3.2
// Attribute Defaults
// [60]
defaultDecl
: Required
| Implied
| (Fixed)? StringLiteral
;




// 3.4
// Conditional Section
// [61]
conditionalSect
: includeSect
| ignoreSect
;

// [62]
includeSect
: Cond_Open Include LBRACKET extSubsetDecl CDataEnd
;

// [63]
ignoreSect
: Cond_Open IGNORE LBRACKET ignoreSectContents* CDataEnd
;

ignoreSectContents
: IgnoreItem ( Cond_Open ignoreSectContents CDataEnd IgnoreItem)*
;



// 4.3
// Text Declaration
// [77]
textDecl : XMLDeclOpen versionInfo? encodingDecl PIClose ;

// Well-Formed External Parsed Entity
// [78]
extParsedEnt
: textDecl? content
;

// Encoding Declaration
// [80]
encodingDecl : Encoding EQ StringLiteral ;



// 4.7
// [82]
notationDecl
: NotationDeclOpen Name (externalID | publicID) GT
;

// [83]
publicID
: PublicToken StringLiteral
;


// 4.2.2 External Entities
// [75]
externalID
: SystemToken StringLiteral
| PublicToken StringLiteral StringLiteral
;

// [76]
nDataDecl
: NData Name
;


// 4.2
// Entity Declaration
// [70]
entityDecl
: EntityDeclOpen PCT? Name entityDef GT
;

// [73]
entityDef
: StringLiteral
| externalID nDataDecl?
;




// 4.2
// Entity Reference
// [67]
reference
: EntityRef
| CharRef
;


// 3.1
// Start-tag

// [40]
sTag : STagOpen (attribute|versionInfo)* GT ;



// [41]
attribute : Name EQ StringLiteral ;

// End-tag
//[42]
eTag : ETagOpen Name GT ;


// Tags for Empty Elements
// [44]
emptyElemTag : STagOpen (attribute)* ETagClose ;




// 2.9
// Standalone Document Declaration
// [32]
sdDecl : Standalone EQ StringLiteral ;


// [28a]
declSep : PEReference ;

// [23]
xmlDecl : XMLDeclOpen versionInfo (encodingDecl)? (sdDecl)? PIClose ;



// [24]
versionInfo : Version EQ StringLiteral ;


// LEXER



// 4.1
// Character Reference
// [66]
CharRef
: '&#' DIGIT+ ';'
| '&#x' ( DIGIT | 'a'..'f' | 'A'..'F' )+ ';'
;


// [68]
EntityRef : '&' Name ';' ;

// [69]
PEReference : '%' Name ';' ;



STagOpen : LT Name ;



ETagOpen : '' ;


DocTypeOpen : '' ;



CommentOpen : '' ;




// Key words and long tokens


CDataEnd : ']]>' ;

ElementDeclOpen : '
// 003F
fragment QUEST : '?' ;

// 0040
fragment AT : '@' ;

// 0041..005A
fragment UPPER : 'A'..'Z' ;

// 005B
fragment LBRACKET : '[' ;

// 005C
fragment BACK_SLASH : '\\';

// 005D
fragment RBRACKET : ']';


// 005F
fragment UNDER_SCORE : '_' ;


// 0061..007A
fragment LOWER : 'a'..'z' ;

// 007D
fragment PIPE : '|' ;

// [3]
// #x20 ' ' \u0020
// #xA '\n' \u000A
// #xD '\r' \u000D
// #x9 '\t' \u0009
WS
: ( '\r' | '\n' | '\t' | ' ' )+ {$channel=HIDDEN;}
;

Wednesday, September 9, 2009

Short rant for this evening.

Java community, can we please, please, please put JSF to rest?

I just spent 1/2 a day today tracking down a cryptic Illegal Syntax for a Setter. Or some other nonsense like that.

Who in world wrote that error message description?

And what does it mean?

In my case I couldn't set a field to null. So why not say something genius like Cannot set Field to Null? Or Illegal Value, null, for Field fieldname.

It wasn't even a syntax issue, it was the value I was passing in. So why mention Syntax in the error message. Brilliant.

Give me back my lean mean JSPs.

Wednesday, September 2, 2009

One more plug before I sign off for the evening.

If you really have a craving for lisp on the JVM.
Take a look at Armed Bear Common Lisp.
http://common-lisp.net/project/armedbear/
Since its all the rage to abandon Java lately. Or to become a polyglot programmer.
I did some due diligence on learning new languages, by taking a look at clojure.

There are some pros and cons. Frankly, the cons are much much larger than the pros right now. Overall all, I'd say its promising. But its really the infancy of clojure. Or even a fetal stage.

Some pluses.
Pretty easy to pick up, if you've ever dealt with lisp in the past.
Lightweight. The clojure jar is very small and unobstrusive.
Functional. Yep, its functional.
Concurrent. This is more of a wash. I'm not seeing any advantage here.
The eclipse plugin works. Bare bones, yes. But it works. (Hint, hint Scala team.)

Now some cons. I would view this as constructive feedback more than dinging the language.
Documentation - not good.
closure-contrib - Its a mess. I see some potential for this to turn into the chaos of CPAN. Please head that off now.
Threading/concurrency. Not really that great or easy. Scala and the actor model are better.
Java interop - for claiming to be one of its strengths. Its not good, especially compared to competitors like Scala, Groovy, etc. The way you code against Java APIs just doesn't feel right. Talk about impedance mismatches. This makes OO and RDBMs seem like peanut butter and jelly.

And probably the biggest road block to adoption. Its lisp. Now if you are lisp person already. It might tempt you. And get you on a JVM platform. However, if you're a lisp user now, you probably wouldn't stoop to the level of Java or the JVM.

On the other hand I don't see it drawing too many non-lisp developers.

One other ding against clojure. Its targeting the JVM, Java interop, and I'm assuming Java developers. The pure functional approach is a steep learning curve. The hybrid OO/functional approach of Scala is much more appealing. Its not going cold turkey to drop OO. You can ease into functional or just use some occasional functional techniques like closures.

Also, I have to say. Is OO really that bad? The industry has developed OO concepts for 30+ years. There must be some wisdom there, right? But like anything else, it can be abused and overused.

As an extended side note. If you're considering clojure because its functional, concurrent, etc.

You should consider taking a look at Haskell and Erlang.
Haskell with the STM has a very strong case for solving concurrency issues. Tthe STM is one of the most interesting concepts I've come across in a while for solving concurrency/threading issues. (Yes, I've had my head in the Java sand for a while.)

And Erlang with the actor model is flat out the best technique I've ever seen for threading and concurrency. It feels natural and its dead simple to pick up.

The other huge pluses with Haskell and Erlang. They read much better than lisp. And not just a slightly better, much much better.

One more plug for Erlang. If you believe in a VM for your languages. Erlang has it. Its a battle tested VM. Telcos have using for years. Not just to process data, but to actually run telco switches. And the financial industry is moving to erlang for market trading apps.

More on Erlang later. I've got some Haskell and Prolog to brush up on.