[XML-DEV Mailing List Archive Home] [By Thread] [By Date] [Recent Entries] [Reply To This Message]

Re: Testing XML don't use xUnit

• From: Ihe Onwuka <ihe.onwuka@gmail.com>
• To: Michael Sokolov <msokolov@safaribooksonline.com>
• Date: Fri, 12 Apr 2013 06:05:51 +0100

On Thu, Apr 11, 2013 at 1:45 PM, Michael Sokolov
<msokolov@safaribooksonline.com> wrote:
> On 04/11/2013 06:37 AM, Stephen D Green wrote:
>
> The problem with Selenium tests is not the tool: it's the markup.  HTML is
> not semantic markup, which means: it's not meaningful.  We can extract
> meaning from it, but its not easy because the underlying application logic
> (which is usually what we want to test) is not what people are thinking
> about when they are crafting the markup.   They're thinking about how to get
> pixels to line up on four different browser platforms.
>

This is something I am involved in at the moment. Crafting an
algorithm to extract semantic meaning from HTML markup. It has not
been too painful to achieve good results so far (using the right
toolset helps)  and it strikes me as a very pragmatic solution to the
Selenium problem. Probably not palatable to the community at large  as
I can see testers arguing that it isn't t kosher to "transform the
GUI" like that. Whereas the real objection would be that  once you've
done that it is no longer a Selenium project (with all that entails).

> Selenium IDE is great for testing layout.  Not as good for testing behavior,
> but in some cases it's all we have.  We've had good success by working with
> the layout developers to get them to include meaningful identifiers in the
> markup *so that it can be tested*.
>

When I look back on that initial project that introduced me to QA
automation I'm amazed at how much the management got right. They soon
figured that it's value was limited to regression testing the GUI
rather than  functionality. Maybe that was because although they were
using a testing tool it was not treated as a testing project. An
inherent part of testing a GUI is testing the look and feel.
Automating that defeats the object. Selenium would tell you
(presumably by failing)  that the layout has changed but it cannot
tell if the layout change was ergonomically beneficial to the user as
that requires human qualities of discernment. So it is not a good fail
(doesn't highlight the need for a code fix).  I suppose if you have
been able to reduce the ergonomics to something that can be
algorithmically expressed - element a should be this much proximate to
element b etc - then it would be automatable. I'll leave the value of
doing that as an unanswered question. I should probably investigate
how it handles the interrogation of  layout that is  represented in
CSS rather than markup as that is going to be a challenge for us.

> If you want to make sure the price calculation is displayed to the user
> properly on a checkout screen, it's much more stable and testable if there
> is an id="price" attribute on the price display box.  Otherwise, if you end
> up having to write XPath like //div[@class="blue-box"]/ul/li[3]/span to
> select the price, you've lost the game.
>

That would be called making the application testable ahhh - I see you
used the very same word. A consideration that devs were not always
sympathetic to hence the existence/prevalence of the latter type of
expression.

• References:
  • Testing XML don't use xUnit
    • From: Ihe Onwuka <ihe.onwuka@gmail.com>
  • Re: Testing XML don't use xUnit
    • From: Andrew Welch <andrew.j.welch@gmail.com>
  • Re: Testing XML don't use xUnit
    • From: Ihe Onwuka <ihe.onwuka@gmail.com>
  • Re: Testing XML don't use xUnit
    • From: Michael Kay <mike@saxonica.com>
  • Re: Testing XML don't use xUnit
    • From: Dimitre Novatchev <dnovatchev@gmail.com>
  • Re: Testing XML don't use xUnit
    • From: Andrew Welch <andrew.j.welch@gmail.com>
  • Re: Testing XML don't use xUnit
    • From: Ihe Onwuka <ihe.onwuka@gmail.com>
  • Re: Testing XML don't use xUnit
    • From: George Cristian Bina <george@oxygenxml.com>
  • Re: Testing XML don't use xUnit
    • From: Ihe Onwuka <ihe.onwuka@gmail.com>
  • Re: Testing XML don't use xUnit
    • From: Andrew Welch <andrew.j.welch@gmail.com>
  • Re: Testing XML don't use xUnit
    • From: Ihe Onwuka <ihe.onwuka@gmail.com>
  • Re: Testing XML don't use xUnit
    • From: Stephen D Green <stephengreenubl@gmail.com>
  • Re: Testing XML don't use xUnit
    • From: Michael Sokolov <msokolov@safaribooksonline.com>