Using dotnet watch test for continuous testing with .NET Core and XUnit.net
When teaching .NET Core I do a lot of "dotnet new" Hello World demos to folks who've never seen it before. That has it's place, but I also wanted to show how easy it is to get setup with Unit Testing on .NET Core.
For this blog post I'm going to use the command line so you know there's nothing hidden, but you can also use Visual Studio or Visual Studio Code, of course. I'll start the command prompt then briefly move to Code.
Starting from an empty folder, I'll make a SomeApp folder and a SomeTests folder.
C:\example\someapp> dotnet new
C:\example\someapp> md ..\sometests && cd ..\sometests
C:\example\sometests> dotnet new -t xunittest
At this point I've got a HelloWorld app and a basic test but the two aren't related - They aren't attached and nothing real is being tested.
Tests are run with dotnet test, not dotnet run. Tests are libraries and don't have an entry point, so dotnet run isn't what you want.
c:\example>dotnet test SomeTests
Project SomeTests (.NETCoreApp,Version=v1.0) was previously compiled. Skipping compilation.
xUnit.net .NET CLI test runner (64-bit win10-x64)
Discovering: SomeTests
Discovered: SomeTests
Starting: SomeTests
Finished: SomeTests
=== TEST EXECUTION SUMMARY ===
SomeTests Total: 1, Errors: 0, Failed: 0, Skipped: 0, Time: 0.197s
SUMMARY: Total: 1 targets, Passed: 1, Failed: 0.
I'll open my test project's project.json and add a reference to my other project.
{
"version": "1.0.0-*",
"buildOptions": {
"debugType": "portable"
},
"dependencies": {
"System.Runtime.Serialization.Primitives": "4.1.1",
"xunit": "2.1.0",
"dotnet-test-xunit": "1.0.0-rc2-*"
},
"testRunner": "xunit",
"frameworks": {
"netcoreapp1.0": {
"dependencies": {
"Microsoft.NETCore.App": {
"type": "platform",
"version": "1.0.1"
},
"SomeApp": "1.0.0-*"
},
"imports": [
"dotnet5.4",
"portable-net451+win8"
]
}
}
}
I'll make a little thing to test in my App.
public class Calc {
public int Add(int x, int y) => x + y;
}
And add some tests.
public class Tests
{
[Fact]
public void TwoAndTwoIsFour()
{
var c = new Calc();
Assert.Equal(4, c.Add(2, 2));
}
[Fact]
public void TwoAndThreeIsFive()
{
var c = new Calc();
Assert.Equal(4, c.Add(2, 3));
}
}
Because the Test app references the other app/library, I can just make changes and run "dotnet test" from the command line. It will build both dependencies and run the tests all at once.
Here's the full output inclding both build and test.
c:\example> dotnet test SomeTests
Project SomeApp (.NETCoreApp,Version=v1.0) will be compiled because inputs were modified
Compiling SomeApp for .NETCoreApp,Version=v1.0
Compilation succeeded.
0 Warning(s)
0 Error(s)
Time elapsed 00:00:00.9814887
Project SomeTests (.NETCoreApp,Version=v1.0) will be compiled because dependencies changed
Compiling SomeTests for .NETCoreApp,Version=v1.0
Compilation succeeded.
0 Warning(s)
0 Error(s)
Time elapsed 00:00:01.0266293
xUnit.net .NET CLI test runner (64-bit win10-x64)
Discovering: SomeTests
Discovered: SomeTests
Starting: SomeTests
Tests.Tests.TwoAndThreeIsFive [FAIL]
Assert.Equal() Failure
Expected: 4
Actual: 5
Stack Trace:
c:\Users\scott\Desktop\testtest\SomeTests\Tests.cs(20,0): at Tests.Tests.TwoAndThreeIsFive()
Finished: SomeTests
=== TEST EXECUTION SUMMARY ===
SomeTests Total: 2, Errors: 0, Failed: 1, Skipped: 0, Time: 0.177s
SUMMARY: Total: 1 targets, Passed: 0, Failed: 1.
Oops, I made a mistake. I'll fix that test and run "dotnet test" again.
c:\example> dotnet test SomeTests
xUnit.net .NET CLI test runner (64-bit .NET Core win10-x64)
Discovering: SomeTests
Discovered: SomeTests
Starting: SomeTests
Finished: SomeTests
=== TEST EXECUTION SUMMARY ===
SomeTests Total: 2, Errors: 0, Failed: 0, Skipped: 0, Time: 0.145s
SUMMARY: Total: 1 targets, Passed: 1, Failed: 0.
I can keep changing code and running "dotnet test" but that's tedious. I'll add dotnet watch as a tool in my Test project's project.json.
{
"version": "1.0.0-*",
"buildOptions": {
"debugType": "portable"
},
"dependencies": {
"System.Runtime.Serialization.Primitives": "4.1.1",
"xunit": "2.1.0",
"dotnet-test-xunit": "1.0.0-rc2-*"
},
"tools": {
"Microsoft.DotNet.Watcher.Tools": "1.0.0-preview2-final"
},
"testRunner": "xunit",
"frameworks": {
"netcoreapp1.0": {
"dependencies": {
"Microsoft.NETCore.App": {
"type": "platform",
"version": "1.0.1"
},
"SomeApp": "1.0.0-*"
},
"imports": [
"dotnet5.4",
"portable-net451+win8"
]
}
}
}
Then I'll go back and rather than typing "dotnet test" I'll type "dotnet watch test."
c:\example> dotnet watch test
[DotNetWatcher] info: Running dotnet with the following arguments: test
[DotNetWatcher] info: dotnet process id: 14064
Project SomeApp (.NETCoreApp,Version=v1.0) was previously compiled. Skipping compilation.
Project SomeTests (.NETCoreApp,Version=v1.0) will be compiled because inputs were modified
Compiling SomeTests for .NETCoreApp,Version=v1.0
Compilation succeeded.
0 Warning(s)
0 Error(s)
Time elapsed 00:00:01.1479348
xUnit.net .NET CLI test runner (64-bit .NET Core win10-x64)
Discovering: SomeTests
Discovered: SomeTests
Starting: SomeTests
Finished: SomeTests
=== TEST EXECUTION SUMMARY ===
SomeTests Total: 2, Errors: 0, Failed: 0, Skipped: 0, Time: 0.146s
SUMMARY: Total: 1 targets, Passed: 1, Failed: 0.
[DotNetWatcher] info: dotnet exit code: 0
[DotNetWatcher] info: Waiting for a file to change before restarting dotnet...
Now if I make a change to either the Tests or the projects under test it will automatically recompile and run the tests!
[DotNetWatcher] info: File changed: c:\example\SomeApp\Program.cs
[DotNetWatcher] info: Running dotnet with the following arguments: test
[DotNetWatcher] info: dotnet process id: 5492
Project SomeApp (.NETCoreApp,Version=v1.0) will be compiled because inputs were modified
Compiling SomeApp for .NETCoreApp,Version=v1.0
I'm able to do all of this with any text editor and a command prompt.
How do YOU test?
Sponsor: Do you deploy the same application multiple times for each of your end customers? The team at Octopus have taken the pain out of multi-tenant deployments. Check out their latest 3.4 release!
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
In my setup, I used a slightly different way to reference the main project from the test project:
{
"version": "1.0.0-*",
"testRunner": "xunit",
"dependencies": {
"dotnet-test-xunit": "2.2.0-preview2-build1029",
"xunit": "2.2.0-beta2-build3300",
"Microsoft.AspNetCore.TestHost": "1.0.0",
"Newtonsoft.Json": "9.0.1",
"Shouldly": "2.8.2",
"AntMeehan.Budget.WebApi": {
"target": "project"
},
"Microsoft.Extensions.Configuration.UserSecrets": "1.0.0",
"NSubstitute": "2.0.0-rc"
},
In fact, I was trying to find a reference, and I found your blog entry from only a few weeks ago :)
Ant - thanks! I need to make sure I know the difference between those two ways.
do someapp and sometests should be in different folders?
This is my first toe in the water with .NET core and I have a couple of basic questions:
1. It didn't look like you did a "dotnet restore" at any point in the process. I had to in order to get the examples to run. Is my environment setup differently?
2. Is there any way to get the output of dotnet test to shut up about the minutia? All I want to see is that everything passed or what tests failed. All the details of compilation are just clutter.
I have gotten NUnit working but I still haven't set up file watch with FAKE because they seem to have quite a few steps for macOS. Maybe I should have actually started with dotnet since I'm really doing experimental stuff anyway.
No executable found matching command "dotnet-watch"
Turns out to be a missing step. If you haven't previously installed the Watcher tool, you'll need to run
dotnet restorein order to make the watch command.
A minor thing, but perhaps that will save someone else a few minutes of head-scratching. ;-)
Works like a dream on my windows environment but am having issues on Mac.
Ran into a known issue when targeting net451 where dotnet test cannot find dotnet-test-xunit.exe
Solved that by having the test project target netcoreapp1.0
Now cannot get the watcher to work.
(dotnet restore gave me Microsoft.DotNet.Watcher.Tools 1.0.0-preview3-final)
The specified framework 'Microsoft.NETCore.App', version '1.1.0-preview1-001100-00' was not found.
I am considering giving .net core a break until .netstandard 2.0 is all ready and the tooling settles down.
Probably needs a mention that the watcher requires the latest and greatest version.
No executable found matching command "dotnet-watch"too, and wasted some time following a trail of suggestions involving alternate ways to specify the tool | dependency | version and to check that the package had been correctly downloaded and installed and that the resulting executable was available.
Turns out my command shell was in the wrong folder!
\example> dotnet test sometestsruns the tests in the sometests project, but
\example> dotnet watch test sometestsfails with this misleading message.
\example> cd sometestsworks as described.
\example\sometests> dotnet watch test
The other week I hooked up VSTS with unit tests for .NET Core (blog post), which now automatically handles all builds and tests for my CI scenarios. It's pretty sweet to get these results (trx) integrated with MSBuild to get the full dashboards in VSTS.
If using tests with the above approach you mention, do you know if the integration scenarios with MSBuild/VSTS/TFS et al works the same way (or is possible)?
Cheers,
Tobi.
Perhaps it would be useful to update the article to reflect this?
Comments are closed.
However, what strikes me the most is that you're not yet using one of the decent open-source assertion libraries like www.fluentassertions.com. Any reason for that?