[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

RE: Understanding Scaling - Critical for a Net-Centric Vision

  • To: "'Roger L. Costello'" <costello@m...>
  • Subject: RE: Understanding Scaling - Critical for a Net-Centric Vision
  • From: "Bullard, Claude L (Len)" <len.bullard@i...>
  • Date: Fri, 20 May 2005 15:15:36 -0500
  • Cc: 'XML Developers List' <xml-dev@l...>

vision mechanism
Systems scale to the energy budget for work performed.   This means:
1.  If the added unit has proportional energy added, it can scale almost
infinitely (not quite infinitely because of interference and maintenance).
2.  Inversely proportional systems that perform no work can scale
almost infinitely as well.   See civil service and consultancies.  :-)
3.  Don't do much but do that precisely and efficiently.
Simpler systems scale because of the low energy requirements,
but the systems that scale best replace energy lost to the environment
from the environment which is why self-sustaining expanding systems
generally have a feedback mechanism for self-regulation.  Thus at a
certain scale, they will stop expanding, collapse, or acquire neighboring
systems for their resources.   If you watch the history of open source
projects, note the exchanges of members, ideas, technologies, and
the merging and fracturing of the communities.   Systems don't
usually have long lifecycles unless they have an active trading
arrangement with their neighbors.   That is why I advise such efforts
to start initially with a fairly closed and focused effort (don't go to
PR immediately; it attracts too many non-contributors); then once
the system is self-regulating and coherent, find the natural allies
with whom one can begin to trade goods and services.
Many efforts fail because they promote the effort before they
have anything substantial to trade.  They are easy prey to
others and themselves.

From: Roger L. Costello [mailto:costello@m...]
Hi Folks,
A few musings about "scaling" which I thought might be of interest ...
Note: In this message I use the word "technology" in its broadest sense.  It can mean a specific tool, or it can mean an approach or strategy.
A key aspect of the Web is its ability to scale.  Web technologies flourish or die depending on their ability to scale.
What does it mean for a technology to scale?  What does it mean that a technology does not scale?
To have a sensible net-centric vision it is critical to have a deep, intuitive understanding of this concept of scaling.  As a company moves to a net-centric environment it must adopt technologies that scale well.  A technology will either scale or it will die.
Let's consider some examples.
Roger Ebert is a professional movie critic.  He reviews a movie and then provides information about it:
   - a short description
   - a rating (1 - 4 stars)
[Note that what Ebert provides is "metadata" about the movie]
Over the years, the number of movies has multiplied (independent movies, blockbuster movies, etc).  It is no longer possible to Ebert to review all them.
Having a single professional movie critic review movies doesn't scale.  Even a group of professional movie critics would be soon overloaded.
As well as not scaling, there is a problem with "truth".  A movie review represents someone's interpretation, not facts.  Everyone has their own opinions about a movie.  Ebert may love the movie, but the rest of the world may hate it.
An alternative to using a professional movie critic to review movies is to have each movie producer generate a movie review.  This approach scales well - there is one movie reviewer for each movie.  Of course, the problem with this approach is that the movie review will be heavily biased.
A third alternative is to record the comments of the movie-goers.  This approach is highly scalable - there are many reviewers for each movie.  Further, since there are many reviewers there is a diversity of opinions, which give a variety of perspective to someone who is trying to decide whether to view a movie.
Thus, a scalable movie review "technology" is one that can grow as the number of movies to be reviewed increases.
The New York Times has a small group of book reviewers.  This group reviews books and then publishes their reviews.
Just as with the problem of reviewing movies, using professional reviewers does not scale.  The New York times group is unable to review the exponentially growing number of new and old books.
An alternative approach is for each book author to elicit a review from a fellow author.  This scales well, but suffers from bias.
A third alternative is to let the users (the readers) provide a review of the books.  This scales well and provides for a diversity of opinions.
A scalable book review "technology" is one that can grow as the number of books to be reviewed increases.
There are some lessons to be learned from the above examples. There are three approaches to solving a problem:
1. The high priests approach: Solve the problem using a small, centralized set of
    skilled professionals (the professionals may be geographically separate but
    functionally they are centralized)
2. The authors approach: Solve a problem using the authors.
3. The users approach: Solve the problem using the large body of distributed users.
The high priests approach is not scalable, although it may produce precise, reliable results.
The author approach is scalable, but it may produce dubious results.
The user's approach is scalable and produces multiple results which are then shared. 
A successful web technology is one that:
    - scales and
    - shares
Technologies that scale invite participation by the users.  When the users become the professionals then the technology has succeeded: 
- When your users style the XML data to their own liking and share their stylesheets with others, then your technology for making appealing looking data has succeeded.
- When your users markup data using tags that make sense to them and share their tags with others, then your markup technology has succeeded.
- When your movie goers write reviews of movies and post them for others to view, then your movie review technology has succeeded.
- When your book readers write review of books and post them for others to view, then your book review technology has succeeded.
A long time ago (Oct. 28, 2003) John Cowan made the below remark on xml-dev.  I now understand it.
Roger L. Costello scripsit:

> BTW John, I really liked your paradigm shift on the telephone operator story.
> Can you think of a similar shift with programming?  /Roger

Yes, in fact.  Perhaps ten years ago, I was sitting next to a stranger
on the New York to Boston shuttle.  In casual conversation, she asked
me what I did.  "I'm a computer programmer", said I.  She said, not
unusually, "Oh, I don't know anything about programming", but then
added in the same breath "except for some Excel and Basic."  I had
an epiphany: the world had changed.
As always, comments are welcome. /Roger


Purchasing Stylus Studio from our online shop is Easy, Secure and Value Priced!

Buy Stylus Studio Now

Download The World's Best XML IDE!

Accelerate XML development with our award-winning XML IDE - Download a free trial today!

Don't miss another message! Subscribe to this list today.
First Name
Last Name
Subscribe in XML format
RSS 2.0
Atom 0.3

Stylus Studio has published XML-DEV in RSS and ATOM formats, enabling users to easily subcribe to the list from their preferred news reader application.

Stylus Studio Sponsored Links are added links designed to provide related and additional information to the visitors of this website. they were not included by the author in the initial post. To view the content without the Sponsor Links please click here.

Site Map | Privacy Policy | Terms of Use | Trademarks
Free Stylus Studio XML Training:
W3C Member
Stylus Studio® and DataDirect XQuery ™are products from DataDirect Technologies, is a registered trademark of Progress Software Corporation, in the U.S. and other countries. © 2004-2013 All Rights Reserved.