[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Prospective vs Retrospective Search (was: RE: Jim Gray article

  • To: "'Jonathan Robie'" <jonathan.robie@d...>, "'XML Developers List'" <xml-dev@l...>
  • Subject: Prospective vs Retrospective Search (was: RE: Jim Gray article on Next Generation Databases)
  • From: "Bob Wyman" <bob@w...>
  • Date: Wed, 4 May 2005 01:09:07 -0400
  • In-reply-to: <4277EFB2.70501@d...>
  • Organization: PubSub Concepts, Inc.
  • Reply-to: <bob@w...>
  • Thread-index: AcVQKRZsEaVO5cjLQdqf8+8Xa2ebfAAOztgA

prospective versus retrospective
Another interesting bit in Gray's article was:
"Publish/subscribe systems contribute further to the challenge by inverting
the traditional data/query ratios, requiring that incoming data be compared
against millions of queries instead of queries being used to search through
millions of records."
	What we're seeing here is completion of the "database" problem to
include prospective search in addition to the traditional retrospective
search implemented by today's relational database systems. A retrospective
system such as a traditional database is focused on evaluating queries
against historical collections of data. A retrospective system answers the
question: "What is known?" It is focused on the past. On the other hand, a
prospective system stores queries instead of data and it evaluates each
newly discovered data object against stored queries. A prospective engine
typically answers questions of the form "Tell me whenever x happens!" and is
focused on searching the future -- not the past. Traditionally, database
engines have had prospective search grafted onto them via mechanisms like
	With the exception of financial market applications which deal with
tremendous volumes of data and have very strict timeliness requirements,
there hasn't been much pressure to improve on prospective matching
capabilities until recently. But today, we're seeing high transaction
volumes and increasing timeliness requirements across all application areas.
The old, tacked-on trigger mechanisms are no longer good enough. This has
resulted in a great deal of "stream processing" research at places like
Stanford, Brown, MIT, Berkeley, U Wisconsin, etc. as well as the
implementation of Internet services like that of PubSub ( http://pubsub.com
) or companies like Stonebraker's new StreamBase.com 
	Just as SQL wasn't able to handle the challenges of "object" data
without extensions, I think we'll have to recognize that SQL -- a language
designed for retrospective search -- is simply not an adequate query
language for prospective search. More extensions will be needed. The same
is, of course, true of XQuery since that language was also designed for use
in retrospective systems and didn't take prospective system requirements
into consideration in its design.
	Gray's paper opens the discussion on a good number of issues -- not
the least of which are the issues related to the prospective search or
matching which is typical of Publish/Subscribe systems. Let's hope it's a
fun and useful discussion.

		bob wyman


Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
First Name
Last Name
Subscribe in XML format
RSS 2.0
Atom 0.3

Stylus Studio has published XML-DEV in RSS and ATOM formats, enabling users to easily subcribe to the list from their preferred news reader application.

Stylus Studio Sponsored Links are added links designed to provide related and additional information to the visitors of this website. they were not included by the author in the initial post. To view the content without the Sponsor Links please click here.

Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.