Cart
[Home] [By Thread] [By Date] [Recent Entries]
roddey@u... wrote: > >Ahh, there's the trick. I believe I have most of a design for an data > structure > >that is fast in memory yet is 'flat' and can have its chunks just written > out or > >read in at any point. It builds on some very old ideas I came up with for > a > >language I designed. When viewed as an interchange format, it may not be > the > >most optimal space wise (although it should be better than XML text) but > trades > >a small amount of space for nearly zero processing overhead. There will > >probably also be a procedure for 'compacting' an object for storage into a > >database or sending over a slow link vs. the 'fast' format usable between > >servers in a cluster. > > > > I think though that this would only hold up as long as you are looking at > XML data as a read-only data source. Once you started doing significant > editing of the data, having a flat structure like that would be more of a > hinderance than a help, would it not? What if I have a 10MB flat buffer and > want to add another child to the second element? This kind of gets into the > quandry that you've nailed in one nail, but now its even harder to nail a > whole raft of others as well as with the more general purpose mechanisms. > > I dunno, if I were thinking along these lines, to keep it reasonably > portable, I'd look at the binary format as a fast serialization mechanism > and at least create native language objects for each one. By the time you > put enough stream format markers and whatnot into the stream to know where > things are, and interpret those during runtime, it might be just as fast to > pay the cost for creating a much more flexible, native object format for in > memory manipulation. I have a way around the modification issue. It's a data structure I call 'elastic memory'. It's really the main reason that I'm going to have to start mostly from scratch. I AM trying to hit a number of nails at once and it won't be easy and I'm not sure I can make it perfect, however I believe I can get close. I'm only worrying about Java at the moment with some allowances for certain restrictions that come into play and typical usage in network protocols. There are a number of situations where serialization just doesn't cut it. As I mention, imagine serializing/deserializing on every method call in a program. I designed some of these mechanisms MANY years ago (about 8 I think) while designing a language after I'd already built a language based on Postscript syntax for a project. In testing the first language, I learned the horrors of 'malloc storms' that happen when you follow a typical design paradigm. My system which allowed a complex application to be represented by meta data would do about 25000 mallocs in a standard run through the app. A Java web server app for a very complex app I just completed with a team does about 150,000 object creations (measured by forcing a garbage collection) in one run through the app. It works amazingly well, but still blows most of it's processing for things that could be avoided. The cool thing is that I found a way to implement it in Java. Thanks for sparring with me! ;-) sdw > xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev@i... > Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/ and on CD-ROM/ISBN 981-02-3594-1 > To (un)subscribe, mailto:majordomo@i... the following message; > (un)subscribe xml-dev > To subscribe to the digests, mailto:majordomo@i... the following message; > subscribe xml-dev-digest > List coordinator, Henry Rzepa (mailto:rzepa@i...) -- OptimaLogic - Finding Optimal Solutions Web/Crypto/OO/Unix/Comm/Video/DBMS sdw@l... Stephen D. Williams Senior Consultant/Architect http://sdw.st 43392 Wayside Cir,Ashburn,VA 20147-4622 703-724-0118W 703-995-0407Fax 5Jan1999 xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev@i... Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/ and on CD-ROM/ISBN 981-02-3594-1 To (un)subscribe, mailto:majordomo@i... the following message; (un)subscribe xml-dev To subscribe to the digests, mailto:majordomo@i... the following message; subscribe xml-dev-digest List coordinator, Henry Rzepa (mailto:rzepa@i...)
|
