[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] What is "large" versus "huge"? Here's the answer ....
Hi Folks, Here's the number that separates large from huge: 8K Allow me to explain. An XML lexer breaks up into tokens the stream of characters comprising an XML document. For example, this stream of characters that we recognize as an XML document: <Document>Hello, world</Document> might be broken up into these tokens: '<' Document '>' Hello, world '</' Document '>' All of those tokens are very short. But there are some XML documents with very long tokens. How long should your tokens be? What are large tokens? What are huge tokens? Practically speaking, if an XML element contains a string, how long should you allow the string to be? The answer is 8K. Tokens less than 8K are large. They are of acceptable length and won't incur performance penalties in the lexer. Tokens greater than 8K are huge. They will incur performance penalties in the lexer. Don't create XML documents with elements containing strings longer than 8K. "Flex" is a tool that auto-generates lexers. Page 49 of the Flex manual [1] says this: Another final note regarding performance: as mentioned in Chapter 7 [Matching], page 14, dynamically resizing yytext to accommodate huge tokens is a slow process because it presently requires that the (huge) token be rescanned from the beginning. Thus if performance is vital, you should attempt to match "large" quantities of text but not "huge" quantities, where the cutoff between the two is at about 8K characters per token. There you have it, the numerical definition of "large" and "huge". /Roger [1] https://epaperpress.com/lexandyacc/download/flex.pdf
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] |
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|