[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Web Services and Quality

• To: jmayer@k...
• Subject: Re: Web Services and Quality
• From: "Chiusano Joseph" <chiusano_joseph@b...>
• Date: Wed, 17 Sep 2003 14:09:24 -0400
• Cc: xml-dev@l...
• Organization: Booz Allen Hamilton
• References: <009801c37d45$b5293d20$7c0a0a0a@SCALA>

<Quote1>
Response time, for example, may be critical to one company, but of much
less importance to a second.
</Quote1>
AND
<Quote2>
So, Agency #1 is interested in fast response times, accurate prices,
ability of the web service to determine distances to where the business
traveller needs to do business, amenities such as computer-friendly
(preferably high-speed) rooms, and so on.
</Quote2>

All great points. I would separate these into 2 types of metrics:
performance metrics (such as response time, of course) and business
metrics (the amenities listed above). In fact, I would rename "business
metrics" to "business criteria" since they are more criteria than pure
measurements.

Kind Regards,
Joe Chiusano
Booz | Allen | Hamilton

Joseph Mayer wrote:
> 
> Of course, the problem that I see with any system like this is trying to
> figure
> out all of the metrics that a consumer may want to see.  Response time, for
> example, may be critical to one company, but of much less importance to a
> second.
> To use your example, let's say I have two travel agencies and a person who
> uses
> web services on their own, with no agency involved.  Agency #1 deals mainly
> with
> business travelers, while agency #2 has mostly retired pleasure travelers as
> its clientele.  The person is one of those programmer-geek types that
> sometimes
> wants to go to programming conferences, and sometimes has to travel
> somewhere to
> support a program that they screwed up on (not that this ever happens to
> me...).
> 
> So, Agency #1 is interested in fast response times, accurate prices, ability
> of
> the web service to determine distances to where the business traveller needs
> to
> do business, amenities such as computer-friendly (preferably high-speed)
> rooms,
> and so on.
> 
> Agency #2 is not as interested in the response time, and their set of
> "necessary"
> values provided by the web services is largely different.  Comfort,
> availability
> of swimming pools, closeness of tourist attractions and availability of
> guided
> tours and such become "core" to their needs, while those are just fluff for
> the
> business travelers.
> 
> And the person may just want to be able to pull down an accurate description
> and
> browse for what they want...
> 
> I guess the end result would kind of be like the star ratings on hotels
> themselves.
> I've stayed at 2-star hotels that I've gone back to year after year, because
> they
> served a particular purpose that I liked, and I've been to 4-star hotels
> that I
> wouldn't stay at again because I just didn't think the continental breakfast
> was
> worth the extra price.  Does that mean that the ratings are wrong, or that
> the
> 4-star hotel was just "fooling" the auditors?  Not necessarily.  It just
> means
> that even within one "industry", there may be a plethora of
> metrics/descriptors
> that need to be looked at.  Perhaps too many for an accurate rating system.
> 
> Joe Mayer
> Information Systems
> Kathrein Inc, Scala Division
> Medford, OR, USA
> 
> > -----Original Message-----
> > From: Chiusano Joseph [mailto:chiusano_joseph@b...]
> > Sent: Wednesday, September 17, 2003 9:54 AM
> > To: Nicolas Toper
> > Cc: Simon St.Laurent; xml-dev@l...
> > Subject: Re:  Web Services and Quality
> >
> >
> > Regarding metrics and measurement: I'm going through the W3C Web
> > Services Architecture (W3C Working Draft 8 August 2003[1]) which
> > presents 5 "architectural models" - one of these models is the
> > "Management Model". This model includes the concept of "management
> > metrics". It states:
> >
> > "The value of the metric captures the information at a point in time.
> > Generally these values are numeric, but may be strings as
> > well. This can
> > be contrasted with Measurements that are calculated with a
> > formula based
> > on metrics, e.g. Average response time during the last hour of
> > execution. The metrics requirements do not enforce any implementation
> > pattern. A managed element should allow any available metrics and
> > measurements to be reported according to configurable time intervals,
> > such as cumulative, sliding window, and interval. A managed
> > element must
> > declare which interval types are supported."
> >
> > So one potential criteria for rating a Web service could be (as the
> > example above suggests) average response time. This could be computed
> > dynamically and updated at regular intervals, according to an
> > advertised
> > method (e.g. cumulative, sliding window, and interval - as in the
> > example above).
> >
> > Kind Regards,
> > Joe Chiusano
> > Booz | Allen | Hamilton
> >
> > [1] http://www.w3.org/TR/ws-arch/
> >
> >
> > Nicolas Toper wrote:
> > >
> > > the web have the same problem :=) if you find an answer
> > you'll do better
> > > than google (which might be a part of the solutions)
> > >
> > > -----Message d'origine-----
> > > De : Chiusano Joseph [mailto:chiusano_joseph@b...]
> > > Envoye : mercredi 17 septembre 2003 17:51
> > > A : Simon St.Laurent
> > > Cc : xml-dev@l...
> > > Objet : Re:  Web Services and Quality
> > >
> > > <Quote>
> > > Determining whose feedback to listen to, while still
> > keeping consumers
> > > free of legal or other retailiation from vendors seems like a bigger
> > > problem than the technical issues.
> > > </Quote>
> > >
> > > Yes - it seems like there would need to be some sort of
> > established and
> > > respected ratings service (much like Arbitron rates TV
> > stations here in
> > > the US(?)). Another possibility is for an open standards consortium
> > > (such as OASIS) to, through a TC, establish a framework for
> > Web Services
> > > rating with wide vendor participation. A large vendor
> > backing would add
> > > further legitimacy to such a framework. Then, the framework could be
> > > adopted by a Web service and the various values provided by that Web
> > > service.
> > >
> > > Of course, that leads to the question of how to get Web service
> > > providers to honestly provide values for their services...
> > >
> > > Kind Regards,
> > > Joe Chiusano
> > > Booz | Allen | Hamilton
> > >
> > > "Simon St.Laurent" wrote:
> > > >
> > > > chiusano_joseph@b... (Chiusano Joseph) writes:
> > > > >Lately I've been thinking about what's around the corner for Web
> > > > >services (whether we're close to that corner yet or not
> > is a separate
> > > > >issue). The notion of dynamic discovery and collaboration of Web
> > > > >services comes to mind (choreography, business
> > processes, etc.) - but
> > > > >I'm also thinking in terms of how a Web service can
> > judge the "quality"
> > > > >of another Web service. My question is: is anyone aware
> > of any current
> > > > >efforts on rating Web services?
> > > >
> > > > I don't have a good answer for this, but have heard
> > several suggestions
> > > > for providing EBay-like feedback on services using
> > UDDI-like approaches.
> > > >
> > > > Determining whose feedback to listen to, while still
> > keeping consumers
> > > > free of legal or other retailiation from vendors seems
> > like a bigger
> > > > problem than the technical issues.
> > > >
> > > > -----------------------------------------------------------------
> > > > The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
> > > > initiative of OASIS <http://www.oasis-open.org>
> > > >
> > > > The list archives are at http://lists.xml.org/archives/xml-dev/
> > > >
> > > > To subscribe or unsubscribe from this list use the subscription
> > > > manager: <http://lists.xml.org/ob/adm.pl>

begin:vcard 
n:Chiusano;Joseph
tel;work:(703) 902-6923
x-mozilla-html:FALSE
url:www.bah.com
org:Booz | Allen | Hamilton;IT Digital Strategies Team
adr:;;8283 Greensboro Drive;McLean;VA;22012;
version:2.1
email;internet:chiusano_joseph@b...
title:Senior Consultant
fn:Joseph M. Chiusano
end:vcard

• References:
  • RE: Web Services and Quality
    • From: Joseph Mayer <jmayer@k...>

• Prev by Date: RE: Web Services and Quality
• Next by Date: RE: Web Services and Quality
• Previous by thread: RE: Web Services and Quality
• Next by thread: Re: Web Services and Quality
• Index(es):
  • Date
  • Thread