[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message] RE: Xqueeze: Compact XML Alternative
Note the copied email from Don Brutzman. There are application solutions, in this case, one for the X3D language. Also note as has been noted on this list before, the encumbered MPEG solution which relies on the Schema as well. len From: Mike Champion [mailto:mc@x...] So, I agree that nobody has shown a good solution here, but I don't agree that it means there there's no problem. There may not BE a good solution outside a given application (another point I believe Sean has made) because the parameters and constraints on them (message size, markup overhead, whether or not the vocabulary is known in advance, whether the "SOAP profile" of XML is in use or whether entity expansion must be supported, what kind of bandwidth and processor speed can be assumed, etc.) are so variable. But that doesn't mean that possible solutions shouldn't be explored, IMHO. ************************************************************************** NPS has been working for a few years on a binary encoding of general XML documents/languages, indended for rapid prototyping of network protocols in support of large-scale virtual environments. This message summarizes the XFSP technical approach and proposes we adapt it to support an extensible binary encoding for X3D. Originally called Dynamic Behavior Protocol, our approach was to define a packet-description language in XML. Such a language might easily be used to define the packet payloads used by network protocols, then parsed by a network handler which did the necessary work of reading & writing protocols conforming to the format. We got this working last fall. An important insight was then realized by Andrzej Kapolka, Ekrem Serin and Don McGregor of NPS. It turns out that XML Schema already supports most of the data-structure information needed for a packet description language. We mapped out an approach for doing so, renaming this work the Cross-Format Schema Protocol (XFSP). It was implemented in November and demonstrated at the IITSEC conference in December 2002. Ekrem performed a valuable test when he took the IEEE DIS Specification, encoded the Entity State PDU (ESPDU) as an XML schema, and created an XFSP reader/writer all within one day. Such DIS implementation work took about a year, originally. A brief XFSP description and detailed slideset are available at http://www.movesinstitute.org/xmsf/IITSEC2002/demos/serin/description.html The current version of XFSP source and Ekrem's draft thesis chapter describing this work is on Sourceforge. Detailed finite state machines on serialization/deserialization are included. Comments are welcome. http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/npsnetv/xfsp It appears that this work is suitable for generation of a binary protocol (and hence binary file format) for X3D. We're proposing it as such a basis. After some scrutiny, we can consider whether how to reopen the deferred RFP for X3D compression. _X3D Binary Encoding based on XFSP_ Goals: - smaller X3D files - faster loading - streaming - all other points in the deferred X3D Binary Requirements and RFP http://www.web3d.org/TaskGroups/x3d/binary/X3dBinaryRFP.html Basic principles: - XFSP reads an XML schema for an arbitrary tagset - XFSP thereby produces a packet reader/writer which can handle any valid document corresponding to the XML schema - Packets are in binary form, using tokens for elements/attributes and serialization of payload data - Unambigous efficient tokenization/serialization/deserialization - This algorithm can be tuned for X3D binary encoding. Further/future benefits: - Since the documents are in XML, they are well structured and suitable for further compression. http://www.research.att.com/sw/tools/xmill has some interesting studies on this topic. Arbitrary documents may grow when converted to XML, but then compress better than the gzipped original. - XML format allows simulataneous use of XML Encryption and XML Signature (i.e. Authentication), both in W3C Recommendation (i.e. approved) status. http://www.w3.org/Encryption http://www.w3.org/Signature - XML Encryption/Signature provide default royalty-free (RF) algorithms but also allow specification of alternates. This technique might again be applied in an X3D binary encoding to allow both open and extensible algorithms for geometry compression, quantization, encryption etc. - XML format allows integration of other metadata, such as Resource Description Framework (RDF) which is of likely interest to CAD3D group and others. http://www.w3.org/RDF - Properly adapted, the binary encoding appears to be suitable both as a file format and also for network delivery (e.g. scene streaming, progressive rendering, event delivery, etc.) - Major performance speedups are expected over gzip during binary scene loading, since the binary structures are directly ready for immediate insertion into scene-graph data structures in memory. No further character-based parsing is needed for numeric fields. - Fixing the token set values assigned to X3D nodes and fields permits immediate streamed deserialization upon commencing loading/receipt. - Suitable for implementation in any network- or file-capable programming language. Example XFSP implementation offered is in Java. Even better performance might be achieved through a tuned X3D-binary implementation, rather than an XFSP implementation capable of general XML. - Tuned X3D-XML implementations might be adapted to handle VRML encoding directly. Alternatively, 1-1 translation to X3D's XML encoding allows corresponding usage by VRML encoding also. - Offered by NPS under royaltee-free terms in accordance with the Web3D Consortium Intellectual Property Rights (IPR) Policy http://www.web3d.org/aboutus/ipr.html Design approach: - XML elements are represented both by open-tag and close-tag tokens. This allows rapid regeneration of the original tree-like document structure. - XML attributes represented by single tokens, in same namespace as elements. - Payload data for attributes and element content is type dependent, occupies predictable lengths between tokens. MF array types are preceded by field-length integers. - Payload compression is type specific, e.g. long integers, floats, doubles, etc. - Further typing may be applied for special geometry-related types like Color, Normal, coordinates, indices, pixelTexture, enumerations, UUIDs (e.g. for CAD3D), etc. This would support specialty compression components on a geometric-type by geometric-type basis. - Quantization tables will likely be allowed. - Separate namespaces for prototype/field names and DEF/IDs. - Use of XML validation and Canonical XML form can ensure that scenes are compressed consistently, e.g. independent of arbitrary whitespace, apostrophe/quotation mark attribute delimitations, etc. http://www.w3.org/TR/xml-exc-c14n Streaming: - Need to investigate whether to support just field payloads, or also stream nodes. - Multiple modes may be necessary for uncorrupted updates of field data: (a) direct replacement (b) progressive, appending to arrays (e.g. pass continuous motion-capture data) (c) buffer until delivered, then swap - Event delivery likely contains regular binary segment consisting of node ID, field token and payload data, along with (optionally honored) timestamp. So, there is a lot here but it appears that a compatible composition of all known X3D binary encoding goals might be achieved. Thanks for discussion and review by Alan Hudson and Justin Couch of Yumetech. http://www.yumetech.com http://www.xj3d.org Thanks too for long-running stellar efforts by Don, Andrzej and Ekrem of the NPS MOVES Institute. http://www.movesinstitute.org Comments welcome. Primary discussion forum is x3d-contributors@w...
|
PURCHASE STYLUS STUDIO ONLINE TODAY!Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced! Download The World's Best XML IDE!Accelerate XML development with our award-winning XML IDE - Download a free trial today! Subscribe in XML format
|