[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Big Data and XML

• From: Arjun Ray <arjun.ray@verizon.net>
• To: <xml-dev@lists.xml.org>
• Date: Sun, 30 Mar 2014 09:37:12 -0400

[Repost: the xml-dev list manager swallowed the earlier version; in
fact, the list seemed down for several days.]
 
On Tue, 25 Mar 2014 20:53:21 -0400, I wrote:

| Someone wants to blat out a terabyte of mostly unchanged data every 
| 15 minutes, in XML?  This is demented.

In fact, there is something very artificial - or perhaps, symptomatic
of goldenhammeritis - in all the examples or use cases or scenarios of
XML for "big data" that I've seen so far.

This particular example is no different:

On Sun, 23 Mar 2014 08:16:00 -0600, Brian Aberle <xmlboss@live.com>
wrote:

: Suppose you have a list of "foos".  Suppose that list is very very 
: long. 1 Terabyte. Suppose that you get updated XML source of that 
: list every 15 minutes.  In the huge list, [...]

A list, as a unitary XML document?  Thousands upon millions upon
gajillions of "foos", all piled into a _single_ XML instance?

For the love of mike, why?  

What was wrong with a _list_ of those gajillions of "foos", each an
XML instance by itself (or in whatever format floats your boat)? 

You could then enrich that _list_ by having each item encapsulated
with meta data - such as a unique id, payload size, etc - to optimize
caching or whatever other high-speed, low-latency, humunous-throughput
requirement you have.

: some "foos" are new, some foos() have been updated. 

What about the ones that have been deleted, or expired, or otherwise
bitbucketed?

If this is state of the art in protocols for "big data", and why XML
is touted as a "solution", I can't say that I'm impressed.

• Follow-Ups:
  • RE: Big Data and XML
    • From: Brian Aberle <xmlboss@live.com>

• References:
  • RE: RFC for XML Object Parsing
    • From: Amelia A Lewis <amyzing@talsever.com>
  • RE: RFC for XML Object Parsing
    • From: "Len Bullard" <cbullard@hiwaay.net>
  • Re: RFC for XML Object Parsing
    • From: Arjun Ray <arjun.ray@verizon.net>