Scott Hanselman

Buckle up kids, this is nuts and I'm probably doing it wrong. ;) And it's 2am and I wrote this fast. I'll come back tomorrow and fix the spelling.

I want to have lots of tests to make sure my new podcast site is working well. As mentioned before, I've been updating the site to ASP.NET Core 2.1.

Here's some posts if you want to catch up:

• Eyes wide open - Correct Caching is always hard
• The Programmer's Hindsight - Caching with HttpClientFactory and Polly Part 2
• Adding Cross-Cutting Memory Caching to an HttpClientFactory in ASP.NET Core with Polly
• Adding Resilience and Transient Fault handling to your .NET Core HttpClient with Polly
• HttpClientFactory for typed HttpClient instances in ASP.NET Core 2.1
• Updating jQuery-based Lazy Image Loading to IntersectionObserver
• Automatic Unit Testing in .NET Core plus Code Coverage in Visual Studio Code
• Setting up Application Insights took 10 minutes. It created two days of work for me.
• Upgrading my podcast site to ASP.NET Core 2.1 in Azure plus some Best Practices
• Using LazyCache for clean and simple .NET Core in-memory caching

I've been doing my testing with XUnit and I want to test in layers.

Basic Unit Testing

Simply create a Razor Page's Model in memory and call OnGet or WhateverMethod. At this point you are NOT calling Http, there is no WebServer.

public IndexModel pageModel;

public IndexPageTests()
{
    var testShowDb = new TestShowDatabase();
    pageModel = new IndexModel(testShowDb);
}

[Fact]
public async void MainPageTest()
{
    // FAKE HTTP GET "/"
    IActionResult result = await pageModel.OnGetAsync(null, null);

    Assert.NotNull(result);
    Assert.True(pageModel.OnHomePage); //we are on the home page, because "/"
    Assert.Equal(16, pageModel.Shows.Count()); //home page has 16 shows showing
    Assert.Equal(620, pageModel.LastShow.ShowNumber); //last test show is #620
} 

Moving out a layer...

In-Memory Testing with both Client and Server using WebApplicationFactory

Here we are starting up the app and calling it with a client, but the "HTTP" of it all is happening in memory/in process. There are no open ports, there's no localhost:5000. We can still test HTTP semantics though.

public class TestingFunctionalTests : IClassFixture<WebApplicationFactory<Startup>>
{
    public HttpClient Client { get; }
    public ServerFactory<Startup> Server { get; }

    public TestingFunctionalTests(ServerFactory<Startup> server)
    {
        Client = server.CreateClient();
        Server = server;
    }

    [Fact]
    public async Task GetHomePage()
    {
        // Arrange & Act
        var response = await Client.GetAsync("/");

        // Assert
        Assert.Equal(HttpStatusCode.OK, response.StatusCode);
    }
    ...
}

Testing with a real Browser and real HTTP using Selenium Standalone and Chrome

THIS is where it gets interesting with ASP.NET Core 2.1 as we are going to fire up both the complete web app, talking to the real back end (although it could talk to a local test DB if you want) as well as a real headless version of Chrome being managed by Selenium Standalone and talked to with the WebDriver. It sounds complex, but it's actually awesome and super useful.

First I add references to Selenium.Support and Selenium.WebDriver to my Test project:

dotnet add package "Selenium.Support"
dotnet add package "Selenium.WebDriver"

Make sure you have node and npm then you can get Selenium Standalone like this:

npm install -g selenium-standalone@latest
selenium-standalone install

Selenium, to be clear, puts your browser on a puppet's strings. Even Chrome knows it's being controlled! It's using the (soon to be standard, but clearly defacto standard) WebDriver protocol. Imagine if your browser had a localhost REST protocol where you could interrogate it and click stuff! I've been using Selenium for over 11 years. You can even test actual Windows apps (not in the browser) with WinAppDriver/Appium but that's for another post.

Now for this part, bear with me because my ServerFactory class I'm about to make is doing two things. It's setting up my ASP.NET Core 2. 1 app and actually running it so it's listening on https://localhost:5001. It's assuming a few things that I'll point out. It also (perhaps questionable) is launching Selenium Standalone from within its constructor. Questionable, to be clear, and there's others ways to do this, but this is VERY simple.

If it offends you, remembering that you do need to start Selenium Standalone with "selenium-standalone start" you could do it OUTSIDE your test in a script.

Perhaps do the startup/teardown work in a PowerShell or Shell script. Start it up, save the process id, then stop it when you're done. Note I'm also doing checking code coverage here with Coverlet but that's not related to Selenium - I could just "dotnet test."

#!/usr/local/bin/powershell
$SeleniumProcess = Start-Process "selenium-standalone" -ArgumentList "start" -PassThru
dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:CoverletOutput=./lcov .\hanselminutes.core.tests
Stop-Process -Id $SeleniumProcess.Id

Here my SeleniumServerFactory is getting my Browser and Server ready.

SIDEBAR NOTE: I want to point out that this is NOT perfect and it's literally the simplest thing possible to get things working. It's my belief, though, that there are some problems here and that I shouldn't have to fake out the "new TestServer" in CreateServer there. While the new WebApplicationFactory is great for in-memory unit testing, it should be just as easy to fire up your app and use a real port for things like Selenium testing. Here I'm building and starting the IWebHostBuilder myself (!) and then making a fake TestServer only to satisfy the CreateServer method, which I think should not have a concrete class return type. For testing, ideally I could easily get either an "InMemoryWebApplicationFactory" and a "PortUsingWebApplicationFactory" (naming is hard). Hopefully this is somewhat clear and something that can be easily adjusted for ASP.NET Core 2.1.x.

My app is configured to listen on both http://localhost:5000 and https://localhost:5001, so you'll note where I'm getting that last value (in an attempt to avoid hard-coding it). We also are sure to stop both Server and Brower in Dispose() at the bottom.

public class SeleniumServerFactory<TStartup> : WebApplicationFactory<Startup> where TStartup : class
{
    public string RootUri { get; set; } //Save this use by tests

    Process _process;
    IWebHost _host;

    public SeleniumServerFactory()
    {
        ClientOptions.BaseAddress = new Uri("https://localhost"); //will follow redirects by default

        _process = new Process() {
            StartInfo = new ProcessStartInfo {
                FileName = "selenium-standalone",
                Arguments = "start",
                UseShellExecute = true
            }
        };
        _process.Start();
    }

    protected override TestServer CreateServer(IWebHostBuilder builder)
    {
        //Real TCP port
        _host = builder.Build();
        _host.Start();
        RootUri = _host.ServerFeatures.Get<IServerAddressesFeature>().Addresses.LastOrDefault(); //Last is https://localhost:5001!

        //Fake Server we won't use...this is lame. Should be cleaner, or a utility class
        return new TestServer(new WebHostBuilder().UseStartup<TStartup>());
    }

    protected override void Dispose(bool disposing) 
    {
        base.Dispose(disposing);
        if (disposing) {
            _host.Dispose();
            _process.CloseMainWindow(); //Be sure to stop Selenium Standalone
        }
    }
}

But what does a complete series of tests look like? I have a Server, a Browser, and an (theoretically optional) HttpClient. Focus on the Browser and Server.

At the point when a single test starts, my site is up (the Server) and an invisible headless Chrome (the Browser) is actually being puppeted with local calls via WebDriver. All this is hidden from to you - if you want. You can certainly see Chrome (or other browsers) get automated, but what's nice about Selenium Standalone with hidden/headless Browser testing is that my unit tests now also include these complete Integration Tests and can run as part of my Continuous Integration Build.

Again, layers. I test classes, then move out and test Http Request/Response interactions, and finally the site is up and I'm making sure I can navigate, that data is loading. I'm automating the "smoke tests" that I used to do myself! And I can make as many of this a I'd like now that the scaffolding work is done.

public class SeleniumTests : IClassFixture<SeleniumServerFactory<Startup>>, IDisposable
{
    public SeleniumServerFactory<Startup> Server { get; }
    public IWebDriver Browser { get; }
    public HttpClient Client { get; }
    public ILogs Logs { get; }

    public SeleniumTests(SeleniumServerFactory<Startup> server)
    {
        Server = server;
        Client = server.CreateClient(); //weird side effecty thing here. This call shouldn't be required for setup, but it is.

        var opts = new ChromeOptions();
        opts.AddArgument("--headless"); //Optional, comment this out if you want to SEE the browser window
        opts.SetLoggingPreference(OpenQA.Selenium.LogType.Browser, LogLevel.All);

        var driver = new RemoteWebDriver(opts);
        Browser = driver;
        Logs = new RemoteLogs(driver); //TODO: Still not bringing the logs over yet
    }

    [Fact]
    public void LoadTheMainPageAndCheckTitle()
    {
        Browser.Navigate().GoToUrl(Server.RootUri);
        Assert.StartsWith("Hanselminutes Technology Podcast - Fresh Air and Fresh Perspectives for Developers", Browser.Title);
    }

    [Fact]
    public void ThereIsAnH1()
    {
        Browser.Navigate().GoToUrl(Server.RootUri);

        var headerSelector = By.TagName("h1");
        Assert.Equal("HANSELMINUTES PODCAST\r\nby Scott Hanselman", Browser.FindElement(headerSelector).Text);
    }

    [Fact]
    public void KevinScottTestThenGoHome()
    {
        Browser.Navigate().GoToUrl(Server.RootUri + "/631/how-do-you-become-a-cto-with-microsofts-cto-kevin-scott");

        var headerSelector = By.TagName("h1");
        var link = Browser.FindElement(headerSelector);
        link.Click();
        Assert.Equal(Browser.Url.TrimEnd('/'),Server.RootUri); //WTF
    }

    public void Dispose()
    {
        Browser.Dispose();
    }
}

Here's a build, unit test/selenium test with code coverage actually running. I started running it from PowerShell. The black window in the back is Selenium Standalone doing its thing (again, could be hidden).

If I comment out the "--headless" line, I'll see this as Chrome is automated. Cool.

Of course, I can also run these in the .NET Core Test Explorer in either Visual Studio Code, or Visual Studio.

Great fun. What are your thoughts?

Sponsor: Check out JetBrains Rider: a cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

About   Newsletter

Hosting By