NuGet Package of the Week: FluentAutomation for automated testing of Web Applications
Last week I was exploring today's varied choices we have for Automated Browser Testing. There's headless WebKit "browsers" like PhantomJS and cloud powered multi-browser testing tools like BrowserStack and SauceLabs.
Selenium is kind of the gold standard and offers not only a lot of "drivers" but also a lot of language bindings with which drive a browser. Sometimes browsers update so fast there can be some version incompatibilities with Selenium, but for the most part it works great once you've settled in.
One option I've been looking at is FluentAutomation. It's a fluent automation API that supports Selenium as well as WatiN along with all their flavors and drivers. Since Fluient supports Selenium, that means you can use the Selenium ChromeDriver, IEDriver, Remote Web Driver or even the headless PhantomJS. FluentAutomation is on GitHub, of course, as well as on NuGet.
FluentAutomation has great (and growing) documentation and has adopted and interesting fluent style for it's API.
Now, not everyone likes a "fluent" API so it may take a while to get used to. Often you'll be doing things over many lines when it's really just one line, for example, this is one line:
I.Open("http://automation.apphb.com/forms")
.Select("Motorcycles").From(".liveExample tr select:eq(0)")
.Select(2).From(".liveExample tr select:eq(1)")
.Enter(6).In(".liveExample td.quantity input:eq(0)")
.Expect
.Text("$197.72").In(".liveExample tr span:eq(1)")
.Value(6).In(".liveExample td.quantity input:eq(0)");
Notice the method chaining as well as the use of CSS selectors.
FluentAutomation also has the cool concept of a PageObject to take your potentially brittle scripts and give them more structure. PageObjects group your actions, expectations, and assertions and let you reuse code when a page appears in multiple tests.
For example you could have a high level test (this is XUnit, but you can use whatever you want):
public class SampleTest : FluentTest {
public SampleTest() {
SeleniumWebDriver.Bootstrap(SeleniumWebDriver.Browser.Chrome);
}
[Fact]
public void SearchForFluentAutomation() {
new BingSearchPage(this)
.Go()
.Search("FluentAutomation")
.FindResultUrl("http://fluent.stirno.com/blog/FluentAutomation-scriptcs/");
}
}
Then you can have separate PageObjects that have your own public methods specific to that page, as well as assertions you can reuse.
public class BingSearchPage : PageObject<BingSearchPage> {
public BingSearchPage(FluentTest test) : base(test) {
Url = "http://bing.com/";
At = () => I.Expect.Exists(SearchInput);
}
public BingSearchResultsPage Search(string searchText) {
I.Enter(searchText).In(SearchInput);
I.Press("{ENTER}");
return this.Switch<BingSearchResultsPage>();
}
private const string SearchInput = "input[title='Enter your search term']";
}
public class BingSearchResultsPage : PageObject<BingSearchResultsPage> {
public BingSearchResultsPage(FluentTest test) : base(test) {
At = () => I.Expect.Exists(SearchResultsContainer);
}
public BingSearchResultsPage FindResultUrl(string url) {
I.Expect.Exists(string.Format(ResultUrlLink, url));
return this;
}
private const string SearchResultsContainer = "#b_results";
private const string ResultUrlLink = "a[href='{0}']";
}
You don't have to be all structure and OO if you don't want. You can just as easily write scripts with FluentAutomation and head in a different direction.
FluentAutomation along with ScriptCS = Automating your Browser with C# Script
I've usually used Python with my Selenium scripts. I like being able to just make a text file and start scripting, then run, debug, continue, all from the command line. It feels simple and lightweight. Creating a DLL and running Unit Tests in C# usually comes later, as I can move faster with a "scripting language."
You can do that with ScriptsCS as it gives you project-less C# that effectively is C# as scripting language. Combine this with FluentAutomation and you've potentially got the best of both worlds.
To install, first you need the Windows apt-get open source equivalent, the oddly-named and -spelled Chocolatey. Then you get ScriptCS and the packages for FluentAutomation.
- Install Chocolatey - one line installation here
- Run "cinst ScriptCS" from your command line to use Chocolatey to install ScriptCS
- Now, get the ScriptCS script packages for FluentAutomation like this:
- scriptcs -install FluentAutomation.SeleniumWebDriver
- scriptcs -install ScriptCs.FluentAutomation
Now, as a quick test, create a folder and put a text file called start.csx in it with just these contents:
var Test = Require<F14N>()
.Init<FluentAutomation.SeleniumWebDriver>()
.Bootstrap("Chrome")
.Config(settings => {
// Easy access to FluentAutomation.Settings values
settings.DefaultWaitUntilTimeout = TimeSpan.FromSeconds(1);
});
Test.Run("Hello Google", I => {
I.Open(http://google.com);
});
Notice how there's no namespace, no classes, no main. It's just a script, except it's using C#. You can change the "Chrome" to "IE" or "Firefox" as well, to play around.
Random: I love this Selenium feature, exposed by FluentAutomation...take screenshot!
// Take Screenshot
I.TakeScreenshot("LoginScreen");
If you don't want ScriptCS, while it can act as a REPL itself, there is also the start of a dedicated FluentAutomation REPL (read–eval–print loop). This is basically a command prompt that lets you explore you app interactively and facilitates building your scripts. You can get the Repl as a Chocolatey package as well and just "cinst FluentAutomation.Repl"
You've got LOTS of choices in the world of automated testing. There's so many choices that there's just no good excuse. Pick a library, pick a language, and start automating your web app today.
Related Links
- Explorative Web application scripting using scriptcs and FluentAutomation video and blog post.
- FluentAutomation homepage
Sponsor: Big thanks to ComponentOne, a division of GrapeCity, for sponsoring the blog this week. Their widely popular .NET control suite, Studio Enterprise contains hundreds of data and UI controls such as grids, charts and reports that offer the functionality, features and support you need for current and future application development. Download your trial today!
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
As an alternative, I created SpecsFor.Mvc. Automated hosting of your ASP.NET MVC app via IIS Express, streamlined wrapper on top of Selenium, integrated SMTP host for testing E-mails, and a strongly-typed, refactor-friendly API for interacting with your app. It's far from perfect; documentation is lacking (completely my fault), and there are still a lot of pain points it doesn't address, but I do feel it leads to much better tests than the string-based alternatives.
Matt - isn't HTML and all IDed elements magic strings?
I love this tool and think it deserves some more center-stage time. Been pushing it for my clients for some time, and now with the inclusion of the PageObject construct built-in as a core feature it took yet another step.
On behalf of the FluentAutomation-users out there I thank you for lifting this up to more peoples attention.
Keep your great blogging up!
Has anyone tried using it with Cucumber/Specflow with any success?
In theory this means changes to the UI are insulated from the feature file and the step definitions. The only changes are in the page object (which is were all the magic strings are used).
SpecsFor.Mvc leverages those same conventions. You can specify the a strongly-typed lambda expression to locate and interact with an element on the page based on a view model (SUT.FindFormFor<RegisterModel>().Field(m => m.Email)), or use a strongly-typed lambda expression to navigate to the URL corresponding to an action on a controller (SUT.NavigateTo<AccountController>(c => c.Register())). Because your view and your specs are using strongly-typed expressions, you get compile-time safety and can refactor freely.
There will still be times when you have to drop down and do something with strings. Like I said, SpecsFor.Mvc isn't perfect. :) But those times should be the exception, not the norm.
What would exactly be the development scenario that FluentAutomation would give a test failure other then dead links?
As far as I can see it does not fail on javascript error's for instance, unless some html is not generated due to Javascript errors.
Also it would not detect if the page would render in purple instead of the desired pink.
And does FluentAutomation also function where most / all of the html is generated from a Javascript framework?
Michael - Lots of our users work with SpecFlow successfully using similar patterns as Mike Thomas. As for AAA, the chaining works in a way to encourage this. For example a single chain cannot expect/assert and then go back into an action. The arrange basically boils down to the Open command (or.. Given I'm on the Login Page), take some actions, assert success or fail conditions.
Matt - SpecsFor.Mvc is a very interesting project. I contemplated building something with a similar method of integrating with the app but wanted to maintain the ability to test *any* web app, not just ones built in our chosen stack. Perhaps a more general lib just capable of retrieving the appropriate selectors from MVC apps would help fill the gap. Interesting times we're in!
Peter -
1. Test failures occur either because an action cannot successfully be taken (broken functionality in page, missing elements, etc) or because you've explicitly built in assertions, like any other code-based test.
2. There are many many reasons a javascript error might be thrown on a page that doesn't directly affect the functionality of the app, so we don't blow up on them.
3. If you have a known class for the page color, you can use I.Assert.Class("pink").On("body"); or something similar and get a failure for that occurrence. Asserts on inline styles coming soon.
4. Because it uses either Selenium or WatiN, FluentAutomation is working with the DOM in whatever state it happens to be in, regardless of where the elements came from. We have users testing giant Single-Page-App deployments, big MVC sites and all the way to Rails apps - the entire spectrum really.
To me, this seems like more of a BDD-style testing framework, more than a TDD... and from what I've seen, in BDD style testing, the AAA model doesn't quite fit, since there are usually too many things happening in an order.
I used Jester to perform some BDD testing in PowerShell, and found the same thing... lots of steps, validation occurring between the steps, etc... all to validate that the process works end-to-end.
Great job on the library.
But I think Peter's point about an assertion that no error happened would be pretty useful. I agree with you that we don't want to fail every test, but an Assert.NoErrors() would be great. Heck, even an Assert.NoWarnings() could be useful.
Can't wait for the inline style assertions.
Keep up the good work.
http://juristr.com/blog/2014/02/nightwatch-test-automation/
And thanks for the blog post on Chocolatey / ScriptCS. I'm really hoping to see a scripting "culture" on Windows that's equatable with what's on Linux systems someday. It's so fun and powerful to be able to write scripts that are doing things other than server admin / command-line interfacing without having to open Visual Studio and create a project :)
Maybe instead of "scripting" language we should say "stand-alone source-code executable".
I too think that this was a strange design decision, but I'm sure that they had good reasons for going down the Expect-route. They have listen to many suggestions of supporting Assert as well and it's not in the tool. Good work OSS-developers!
I've used SpecFlow and FluentAutomation together quite a lot. I like it a lot. Here's a blogpost I wrote awhile back: http://www.marcusoft.net/2012/05/specflow-page-objects-and.html and here http://www.marcusoft.net/2013/04/PushTheHowDown.html
Please notice that this is based on a version that didn't have the PageObject support in FluentAutomation so I had to write them myself.
What I find interesting is the way you spin up IIS Express to be able to test a website. Are you also using this in a TeamCity Continuous Integration/Delivery scenario.
https://github.com/leblancmeneses/RobustHaven.IntegrationTests
Why did I build the above? - answer on stackoverflow: http://stackoverflow.com/questions/20860148/looking-for-a-open-source-web-testing-automation-framework/20871839#20871839
There is selenium, kendo, and angularjs extensions that can be used independently.
As seen on nuget: http://www.nuget.org/profiles/leblancmeneses/
As you might be aware that very few test automation projects achieve desired success. In many organizations, Test Automation starts with lot of buzz and potential to reduce the test efforts drastically and optimize costs magically, but in reality more than often it ends up in spending unreasonably high amount of efforts in maintaining the developed automation suite not to mention requirement of highly skilled programmers (Java / C# / VB etc.) for developing automation scripts and the higher development time required to write automation scripts.
Following up on this, I came across and registered for a webinar on Simplifying Test Automation Using Automation Framework http://j.mp/1ivCoe4 which addresses above mentioned challenges, it looks a promising one.
Comments are closed.
Apart from this, and more importantly:
That's quite invalid.
PageObjects as a pattern is mentioned in Selenium's documentation
https://code.google.com/p/selenium/wiki/PageObjects
Also, mentioned by Martin Fowler:
http://martinfowler.com/bliki/PageObject.html
And utilized in other frameworks that build on top of Selenium, like:
http://docs.teststack.net/Seleno/PageObjects.html