Scott Hanselman

OK, I'm hooked. Ruby is the bomb and I'm loving it. We shall see what the future brings, the debugger is iffy, I need to recompile Notepad2 like Wes did with Ruby syntax support, but otherwise I'm digging it.

The back story is this. I've been trying to find the Holy Grail, or even the 40% Grail of Automated Web UI Testing for at least the last 5 years. We use NUnit for everything, so it'd be cool if whatever we chose could fail NUnit tests.

Lately folks have been trying again all over the next and lots of good stuff has been happening in this space.

• NUnitASP - http://nunitasp.sourceforge.net/
  • Pros: C# class library for downloading and parsing web pages. Very CSharpy.
  • Cons: Nothing has happened with this since November 2004. Have to extend it constantly for more complex controls. Thinks about things in a Control way.
• SAMIE - http://samie.sourceforge.net/
  • Pros: Uses Perl to drive IE around using it's OLE automation interface. Keeps logs, can fill with data from databases, and there's a cheesy but functional WinForms app to help write the scripting language. 
  • Cons: Uses Perl. LOL.
• HttpUnit - http://httpunit.sourceforge.net/
• Pros: Feels good to the Java folks, integrates with JUnit, lower level focused.
• Cons: May not be actively developed, no activity since October 2004.
• IEUnit - http://ieunit.sourceforge.net/
• Pros: Almost zero installation footprint. Javascript based engine uses the language we were all born knowing. Rich debugging because you can use the Windows Script Debugger. Fairly simple in design. Actively developed. Nice WinForms script helper. Good if you already dig the DOM.
• Cons: Slightly cheesy handling of model dialogs. Steals focus while running tests.
• Selenium - http://selenium.thoughtworks.com/
  • Pros: Good for testing cross browser compatibility. Actively worked on. Selenium tests can be expanded to use other languages.
  • Cons: Uses a Fit-like HTML as it's source code. This takes a mind shift and freaks out some folks. Server-side component required.

And the one I'm currently enamored with is Watir. There's a great comparison of Watir and Selenium by a contributor to both here.:

• Watir (Pronounced "Water") Web Application Testing in Ruby - http://wtr.rubyforge.org/
  • Pros: Actively being worked on. Uses Ruby, the language du jour. Can run tests asychronously. Simple, elegant, fast. Ruby's interactive shell makes it easy to develop tests (more on this later in this post). Great support for client side things like script events, and even mouse rollovers.
  • Cons: Haven't found any yet, but I'm sure they are out there. Very focused on the DOM.

Ok, just to level set if you made it this far: Watir is an open source functional testing library for automated tests to be developed and run against a web browser. Cool?

Here's an example that visits Google, searches for "Programming Ruby" and confirms that it found what it was looking for. The stuff after the # are comments.

require 'watir' # the watir controller
include Watir
test_site = 'http://www.google.com'
ie = IE.new
ie.goto(test_site)
ie.text_field(:name, "q").set("pickaxe") # q is the name of the search field
ie.button(:name, "btnG").click # "btnG" is the name of the Search button
if ie.contains_text("Programming Ruby")
  puts "Test Passed. Found the test string: 'Programming Ruby'. Actual Results match Expected Results."
else
  puts "Test Failed! Could not find: 'Programming Ruby'"
end

Very simple. So, Patrick and I wanted to integrate this into our NUnit tests, some of which already use Cassini for self-contained tests, and others are more complex but ultimately result in a web site. We wanted to do something like this from our existing infrastructure:

[TestFixture]

    public class WatirTest 

    {

        public WatirTest(){}

        [Test]

        public void FirstTest()

        {

            WatirAssert.TestPassed("ibcexample.rb");

        }

    }

This was the Goal. We talked about adding [WatirTest] attributes or extending a new WatirTestCase and even a whole new series of Assertions, but we didn't see any reason to be tricky, other than to be tricky. We opted for simple simple simple.

public class WatirAssert 

{

    public static void TestPassed(string rubyFileName)

    {

        string output = String.Empty;

        using(Process p = new Process())

        {

            p.StartInfo.UseShellExecute = false;

            p.StartInfo.RedirectStandardOutput = true;

            p.StartInfo.FileName = "ruby.exe";

            p.StartInfo.Arguments = rubyFileName;

            p.StartInfo.WorkingDirectory = AppDomain.CurrentDomain.BaseDirectory;

            p.Start();

            output = p.StandardOutput.ReadToEnd();

            p.WaitForExit();

        }

        Console.Write(output);

        Trace.Write(output);

        Regex reg = new Regex(@"(?<tests>\d+) tests, (?<assertions>\d+) assertions, 
                 (?<failures>\d+) failures, (?<errors>\d+) errors",

            RegexOptions.Compiled);

        Match m = reg.Match(output);

        try

        {

            int tests = int.Parse(m.Groups["tests"].Value);

            int assertions = int.Parse(m.Groups["assertions"].Value);

            int failures = int.Parse(m.Groups["failures"].Value);

            int errors = int.Parse(m.Groups["errors"].Value);

            if (tests > 0 && failures > 0) 

            {

                Assert.Fail(String.Format("WatirAssert: Failures {0}", failures));

            }

            else if (errors > 0) 

            {

                Assert.Fail(String.Format("WatirAssert: Errors {0}", errors));

            }

        }

        catch(Exception e)

        {

            Assert.Fail("WatirAssert EXCEPTION: " + e.ToString());

        }

    }

}

Using this, and armed with increasing knowledge of Ruby we started writing test cases (one per use case) for a large bank as a good test of this concept. These tests would run at the very end of the build process. After the build, test, docs, etc, we'd run the smoke (or integration) tests using Watir.

UPDATE: Your Ruby classes have to derive from the Ruby Unit Testing framework for this to work automatically. So your "template" would be (Note: the class name has an initial capital letter):

require 'watir'
include Watir
require 'test/unit'

class TestExampleTemplate < Test::Unit::TestCase 
  def test_search
    ie = IE.new
    ie.goto("http://www.google.com")
    ie.text_field(:name, "q").set("pickaxe")
    ie.button(:value, "Google Search").click
    assert(ie.contains_text("Programming Ruby, 2nd Ed."))
  end
end

Now, referring back to the syntax:

ie.text_field(:name, "q").set("pickaxe") # q is the name of the search field
ie.button(:name, "btnG").click # "btnG" is the name of the Search button

You may think it'll be a hassle to do a lot of View|Source work and figure out the names and IDs of these textboxes and buttons. However, even though Watir only runs IE (remember it wraps IE's COM stuff) FireFox presented a great way to avoid all that poking through HTML source. Yes, it's the greatest thing FireFox brought to the Web Developer (besides Venkman), it's the Web Developer Extension.

Bring up your site in FireFox and from the Web Developer toolbar, click Forms|Display Form Details. Notice as in the screen shot below you'll get the IDs and Names of all your form elements. Then code your Watir script to them. Shiny.

If you want get Ruby and Watir and start playing:

• Download Ruby for Windows
• Download Watir and run install.rb
• Go into C:\watir_bonus\examples and run googleSuite_test.rb
• Then, look at articleExample.rb and make some changes to test your own site.
• Enjoy. Feel free to integrate into your NUnit tests if you like using our code above. If you expand it, let us know.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By