Azure DevOps Continuous Build/Deploy/Test with ASP.NET Core 2.2 Preview in One Hour
I've been doing Continuous Integration and Deployment for well over 13 years. We used a lot of custom scripts and a lovely tool called CruiseControl.NET to check out, build, test, and deploy our code.
However, it's easy to get lulled into complacency. To get lazy. I don't set up Automated Continuous Integration and Deployment for all my little projects. But I should.
I was manually deploying a change to my podcast website this evening via a git deploy to Azure App Service. Pushing to Azure this way via Git uses "Kudu" to actually build the site. However, earlier this week I was also trying to update my site to .NET Core 2.2 which is in preview. Plus I have Unit Tests that aren't getting run during deploy.
So look at it this way. My simple little podcast website with a few tests and the desire to use a preview .NET Core SDK means I've outgrown a basic "git push to prod" for deploy.
I remembered that Azure DevOps (formerly VSTS) is out and offers free unlimited minutes for open source projects. I have no excuse for my sloppy builds and manual deploys. It also has unlimited free private repos, although I'm happy at GitHub and have no reason to move.
It usually takes me 5-10 minutes for a manual build/test/deploy, so I gave myself an hour to see if I could get this same process automated in Azure DevOps. I've never used this before and I wanted to see if I could do it quickly, and if it was intuitive.
Let's review my goals.
- My source is in GitHub
- Build my ASP.NET Core 2.2 Web Site
- I want to build with .NET Core 2.2 which is currently in Preview.
- Run my xUnit Unit Tests
- I have some Selenium Unit Tests that can't run in the cloud (at least, I haven't figured it out yet) so I need them skipped.
- Deploy the resulting site to product in my Azure App Service
Cool. So I make a project and point Azure DevOps at my GitHub.
They have a number of starter templates, so I was pleasantly surprised I didn't need manually build my Build Configuration myself. I'll pick ASP.NET app. I could pick Azure Web App for ASP.NET but I wanted a little more control.
Now I've got a basic build pipeline. You can see it will use NuGet, get the packages, build the app, test the assemblies (if there are tests...more on that later) and the publish (zip) the build artifacts.
I then clicked Save & Queue...and it failed. Why? It says that I'm targeting .NET Core 2.2 and it doesn't support anything over 2.1. Shoot.
Fortunately there's a pipeline element that I can add called ".NET Core Tool Installer" that will get specific versions of the .NET Core SDK.
NOTE: I've emailed the team that ".NET Tool Installer" is the wrong name. A .NET Tool is a totally different thing. This task should be called the ".NET Core SDK Installer." Because it wasn't, it took me a minute to find it and figure out what it does.
I'm using the SDK Agent version 2.22.2.100-preview2-009404 so I put that string into the properties.
At this point it builds, but I get a test error.
There's two problems with the tests. When I look at the logs I can see that the "testadapter.dll" that comes with xunit is mistakenly being pulled into the test runner! Why? Because the "Test Files" spec includes a VERY greedy glob in the form of **\*test*.dll. Perhaps testadapter shouldn't include the word test, but then it wouldn't be well-named.
**\$(BuildConfiguration)\**\*test*.dll
!**\obj\**
My test DLLs are all named with "tests" in the filename so I'll change the glob to "**\$(BuildConfiguration)\**\*tests*.dll" to cast a less-wide net.
I have four Selenium Tests for my ASP.NET Core site but I don't want them to run when the tests are run in a Docker Container or, in this case, in the Cloud. (Until I figure out how)
I use SkippableFacts from XUnit and do this:
public static class AreWe
{
public static bool InDockerOrBuildServer {
get {
string retVal = Environment.GetEnvironmentVariable("DOTNET_RUNNING_IN_CONTAINER");
string retVal2 = Environment.GetEnvironmentVariable("AGENT_NAME");
return (
(String.Compare(retVal, Boolean.TrueString, ignoreCase: true) == 0)
||
(String.IsNullOrWhiteSpace(retVal2) == false));
}
}
}
Don't tease me. I like it. Now I can skip tests that I don't want running.
if (AreWe.InDockerOrBuildServer) return;
Now my tests run and I get a nice series of charts to show that fact.
I have it building and tests running.
I could add the Deployment Step to the Build but Azure DevOps Pipelines includes a better way. I make a Release Pipeline that is separate. It takes Artifacts as input and runs n number of Stages.
I take the Artifact from the Build (the zipped up binaries) and pass them through the pipeline into the Azure App Service Deploy step.
Here's the deployment in progress.
Cool! Now that it works and deploys, I can turn on Continuous Integration Build Triggers (via an automatic GitHub webhook) as well as Continuous Deployment triggers.
Azure DevOps even includes badges that I can add to my readme.md so I always know by looking at GitHub if my site builds AND if it has successfully deployed.
Now I can see each release as it happens and if it's successful or not.
To top it all off, now that I have all this data and these pipelines, I even put together a nice little dashboard in about a minute to show Deployment Status and Test Trends.
When I combine the DevOps Dashboard with my main Azure Dashboard I'm amazed at how much information I can get in so little effort. Consider that my podcast (my little business) is a one-person shop.
And now I have a CI/CD pipeline with integrated testing gates that deploys worldwide. Many years ago this would have required a team and a lot of custom code.
Today it took an hour. Awesome.
I check into GitHub, kicks off a build, tests, emails me the results, and deploys the website if everything is cool. Of course, if I had another team member I could put in deployment gates or reviews, etc.
Sponsor: Copy: Rider 2018.2 is here! Publishing to IIS, Docker support in the debugger, built-in spell checking, MacBook Touch Bar support, full C# 7.3 support, advanced Unity support, and more.
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
So now that you have all that working, Are you going to convert it to yaml?
I'm working on an improvement to NuKeeper which will allow the build of A to be triggered automatically and presented as a pull request to the main project.
Would this be a deployment gate after deploying to integration (the first stage)?
In theory I would say it is damn too late to run these tests, otherwise you introduced release blockers to your branch which you are using for releases...
You would want to test these as early as possible and that is when creating the pull request. This is why I played around with pull request validation (build policy) which does pre-merging of the changes and runs a clone of the CI build (which does not trigger a release pipeline). But obviously... you would not want to deploy anything there either to run your integratio/API tests.
Is TestServer, in-memory database and "local" build server tests the solution? Don't think this would work out in big projects?
I am still thinking about this stuff, so if somebody has some links regarding these concept... it would be greatly appreciated. ;-)
I just went through the process of adding Selenium UI Tests to a Azure DevOps release recently. If you include the nuget package 'Selenium.WebDriver.ChromeDriver' it will copy the ChromeDriver into your build folder and you will be able to reference it in your tests.
browser = new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location))
If you are only running dotnet core I found that the dotnet task with the test argument was faster than using the VsTest task but both will give you the results you are looking for. That should be all you need to run your Selenium UI Tests via Azure Pipelines. I used nunit for my tests but xunit should run essentially the same.
So when you build, and create a publish package on local machine less then 1 minute,
then it will run more then 10 minutes on VSTS.
So be careful with those free minutes :D
I have set up something similiar for a large web app, the only issue I have is that the number of unit tests I have is >1000 so I get a warning during testing and I'm unable to view the test results in VSTS.
Just something for you and your readers to bear in mind.
:)
We appreciate the feedback on perf, we are doing a lot to speed up our cloud-hosted build agents. Also want to point out that open source projects have "unlimited build minutes".
Comments are closed.