Scott Hanselman

browse by category or date

Web Deployment Made Awesome: If You're Using XCopy, You're Doing It Wrong

March 24, 2010 Comment on this post [48] Posted in ASP.NET | IIS | Mix | Speaking
Sponsored By

Web Deployment Made Awesome: If You're Using XCopy, You're Doing It Wrong I did three talks at Mix 10 this year, and I'm going to do blog posts for each one, sharing what I talked about and some code if it's useful.

I did a talk on Deployment called "Web Deployment Made Awesome: If You're Using XCopy, You're Doing It Wrong."

You can download the talk here, or watch it online:

VIDEO Download: MP4 Video, Windows Media Video, Windows Media Video (High)

I always try to sneak cooler titles into conferences if I can. It's better than "WEB101: Deploying Websites using Microsoft Visual Studio 2010's WebDeploy OneClick Publish Wizard Super Karate Monkey Death Car September CTP R2." Well, maybe not way better, but still.

imageHere's an outline of what Deployment Related topics I tried to cover

  • Web Packaging - Offline vs. Online
    • From VS 2010
    • From IIS Manager
  • Web.Config Transformation
    • Transform Syntax
    • Locator Syntax
    • Why not XSLT?
  • Deploying
    • Command Line
    • What If Switch
    • From IIS
    • Content Sync
  • DB Deployment
    • Scripting Source DB
    • Adding custom SQL Scripts
  • Download and Deployment of Open Source
    • Web PI
    • App Gallery
  • One Click Publish
    • Using Web Deploy (Ms Deploy) WMSVC
    • Using Web Deploy (Ms Deploy) Remote Agent
    • Using InProc Web Deploy (Ms Deploy)

Here's some cool highlights about WebDeployment in Visual Studio 2010. You can right-click on your web.config and click "Add Config Transforms."

When you do this, you'll get a web.debug.config and a web.release.config. You can make a web.whatever.config if you like, as long as the name lines up with a configuration profile. These files are just the changes you want made, not a complete copy of your web.config.

You might think you'd want to use XSLT to transform a web.config, but while they feels intuitively right it's actually very verbose.

Here's two transforms, one using XSLT and the same one using the XML Document Transform syntax/namespace. As with all things there's multiple ways in XSLT to do this, but you get the general idea. XSLT is a generalized tree transformation language, while this deployment one is optimized for a specific subset of common scenarios. But, the cool part is that each XDT transform is a .NET plugin, so you can make your own.




             
    
  


  
    
    
      NewSetting
      New Setting Value

Or the same thing via the deployment transform:

Now, to express this concretely, here's my new NerdDinner web.debug.config:

and here's the web.release.config. Note that I update connectionStrings, change appSettings (in this case, Twitter library stuff), and change the system.web section, turning on customErrors and removing the debug attribute.







  
    
    
  

  
              xdt:Transform="SetAttributes" xdt:Locator="Match(key)" />
              xdt:Transform="SetAttributes" xdt:Locator="Match(key)" />
  
  
  

               xdt:Transform="Replace">

This is just config transforms, which is a small part of the whole deployment process. I also showed the packaging process and the package deployment that can happen with one-click from within Visual Studio, or can be initiated from IIS, or at the command-line from your Continuous Integration solution.

The WebDeploy packaging and deployment solution is also what the Web Platform Installer uses. It's all the same engine. This screenshot is me importing an open source application directly from a zip file. Note it's more than just what files to use, it's also setting ACLs (Access Control Lists, or permissions) and creating an IIS application. This just scratches the surface.

Deploying DasBlog from IIS using WebDeploy 

Check out my talk, I hope it helps you out. There's more content in that 60 min talk than I could easily put in a single blog post.

Related Links

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
Comment on this post [48]
Share on Twitter or Facebook or use the
March 24, 2010 22:44
It would be really awesome if the team created a TFS 2010 Build workflow activity to encapsulate the web deploy functionality to use during the new WF-based automated builds. Hopefully they can get it to where it's just in the box for vNext but I bet people would love to have it loose with an out-of-band release in the meantime.
Ed Blankenship
March 24, 2010 22:52
+10 Ed Blankenship
MotoWilliams
March 24, 2010 23:00
Behind the scenes Web Deployment features are all implemented as MSBuild tasks + targets so TFS 2010 integration comes with it... I will try to write down a blogpost which covers the details, but in the meantime you can also view the video http://vishaljoshi.blogspot.com/2009/11/web-deployment-painkillers-vs-2010-ms.html which actually covers TFS demo...
Thanks
Vishal Joshi
Vishal R. Joshi
March 24, 2010 23:12
I agree with Ed and Moto, a TFS 2010 build task for web deploy would be awesome. I know you can use PSExec and invoke it from the command line, but I want the "easy button" dang it. We could build a community contribution, but seeing as MS will most likely release something (hopefully sooner rather then later), our time would be better spent writing tasks for other community tools (i.e. XUnit).
Adam Salvo
March 24, 2010 23:20
Ah, but I'm not talking about an MSBuild task. Team Build in TFS 2010 is now implemented using Windows Workflow Foundation. This means that we now use actual workflow activities that are part of the build workflow process to perform activities outside of compilation (like deploy.)

(The old MSBuild way of doing things is still around for backwards compatibility but it's recommended to do things that are non-compilation using workflow activities and not MSBuild tasks. People will be using the new workflow-based builds going forward.)
Ed Blankenship
March 24, 2010 23:21
If you are curious about what I mean by a custom workflow activity in TFS 2010 Build, Jim Lamb's got a great blog post available here: How to Create a Custom Workflow Activity for TFS Build 2010
Ed Blankenship
March 24, 2010 23:29
I enjoyed your MSDeploy talk, Scott, and MIX 2010 as a whole! I noticed PowerShell cmdlets for IIS 7.5 and was wondering if maybe you could go over those. You talked a little bit about PowerShell in your talk, and I would love to find out more about it.
Joe Chung
March 24, 2010 23:53
Nice to see they took the ideas from XmlMassUpdate (http://blogs.microsoft.co.il/blogs/dorony/archive/2008/01/18/easy-configuration-deployment-with-msbuild-and-the-xmlmassupdate-task.aspx) and productized it - it was a useful task in need of some love.
Joshua Flanagan
March 25, 2010 0:03
Sounds good for the hobbyist - but - anyone who works for a corporation and who writes and maintains code generally doesn't have the right to publish. I would also submit that people who do the deploying, the last thing they would want to do is go into Visual Studio.

What command line options are there that include auditing a website move?
- You know for SOX compliance?
Dave
March 25, 2010 0:19
I must say that I want this for ages !!!

This is awsome! it just gives me a solid place in my office as I was having so much trouble deploying our Web Apps !!! to you Scoot and to the Web Deploy / IIS Team: BIG Thank you!

have a question though ... is IIS 6 supported or this is only for 7+?
balexandre
March 25, 2010 0:38
I just got done watching the video that Vishal linked. The relevent sections for working with TFS are at 22:15 (build a package and send it to a drop location) and 59:30 (publish from team build). I had known about the /p:DeployOnBuild parameter before, but at 59:30 the other parameters are shown which are:

/p:DeployTarget=MsDeployPublish
/p:CreatePackageOnPublish=True
/p:MSDeployPublishMethod=InProc
/p:MSDeployServiceURL=localhost
/p:DeployIISAppApth="Default Web Site"

I havn't tried it out yet, but it looks like a step in the right direction.


@Dave: Take a look at the video that Vishal linked. It does a pretty good job showing off some of the features that could be used in an enterprise environment, including command line execution. It might not be perfect, but looks like it's a good starting place.
Adam Salvo
March 25, 2010 2:49
Dave - this is totally for more than the hobbyist. This is for large systems. When you don't have direct access to a machine, you create a package, then get that package to IT however you like, then they can import it directly or deploy it from their network. WebDeploy supports both online (publish) and offline (package) scenarios.

Also, it creates a log (and you can also do -whatif) that you can use for Auditing.
Scott Hanselman
March 25, 2010 3:54
what he said =>> +10 Ed Blankenship !

@Scott :: thank you so much for doing a MIX/video on this topic. I feel that this is one of the toughest steps in the entire development process (currently) and the new features that are now part of VS2010 are a massive step to helping us :)

I just hope we can get more video's about this new feature, so we can see edge cases or advanced cases (web farm deployment, etc)

This has been my top wish list item for eons.

/me bows low.
PK
March 25, 2010 10:21
I really like what MSDeploy has to offer but unless I can run it via either MSBuild on the commandline, or MSDeploy from a command line I'm going to be stuck with XCopy deployments :P
https://www.google.com/accounts/o8/id?id=AItOawlu32w3Pha0bsc8OvLHg6081x4zx2L91n0
March 25, 2010 10:30
Great session, Scott. I learned a lot. Love, love the fact that you can import a package directly into IIS using IIS manager. And the web.config transforms are fantastic. Thanks again.
https://www.google.com/accounts/o8/id?id=AItOawnRsz5AS1yEzHeqfhbT4DoznhIZL6VvC5o
March 25, 2010 10:38
To be clear, you CAN use MSDeploy from:

* the command line
* your own apps via API
* vs2010
* IIS manager
* msbuild, etc, etc...


Better?
Scott Hanselman
March 25, 2010 12:08
Any blogs/ articles which cover how to do it from a command line?
https://www.google.com/accounts/o8/id?id=AItOawlu32w3Pha0bsc8OvLHg6081x4zx2L91n0
March 25, 2010 13:24
Much better then web.config replacement.

So I asked this before on another MSDN blog, but I didn't get an answer: who came up with XML Document Transform, why didn't anybody come up with it before and is it a standard in the way XML and XSLT are?

Thanks!
Mike
March 25, 2010 13:53
Nice article Scott. I've recently posted an article on an alternative deployment technique using NAnt, Teamcity, Powershell and Git which is proving to be a robust solution where I work.

Details hereL
http://thecodedecanter.blogspot.com/2010/03/one-click-website-deployment-using.html
http://thecodedecanter.blogspot.com/
March 25, 2010 17:50
I have two questions about Web Deploy:

1. Can I deploy an upgrade with zero downtime?
2. Can I roll back immediately with zero downtime as well?

I've always used XCopy deployment to a separate directory for each version, and changed the settings in IIS to point to the new version to switchover. Unless you have complex configuration or database changes, the downtime you get from this method is, to all intents and purposes, zero.

I'd need to know that Web Deploy could also handle this requirement before I gave it serious consideration.
James McKay
March 25, 2010 19:42
Sweet feature.

Though as soon as I saw I it I wondered if it also works with app.config? I played around with VS2010RC a bit and it appears the answer is no.

Is there any way to make it work with app.config's?
http://www.google.com/profiles/RobichaudBrett
March 25, 2010 22:25
Nice NewsRadio reference ;)
https://www.google.com/accounts/o8/id?id=AItOawmj2tBFh7HQL3BtB9G6lFRi_tVZoREWbmY
March 25, 2010 22:55
My problem with the web.config transform mechanism is that changing things like connection strings, etc. is not a build-time configuration for me (and most people I've talked to), it's a deploy-time configuration. I'm not going to create a different build for dev, integration, test, production, and whatever other environments I might have. The web deploy stuff is nice, but VS needs to incorporate the idea of deploy-time configuration changes -- that's more valuable to me than build-time configuration changes.
Jason Dossett
March 26, 2010 6:17
Excellent post Scott and I really enjoyed the video. However I did run into an issue I haven't seen mentioned before relating to symbols in app settings. I've briefly written about this in Visual Studio 2010 Config Transformations TransformXml task failure which I hope saves anyone else in a similar position some headaches.
http://troyhunt.myopenid.com/
March 30, 2010 21:51

1. Can I deploy an upgrade with zero downtime?


I doubt this will affect your scenario, if you truely do need zero downtime, then you need to load balance, when deploying, have the load balancer point at one server only until the other is up and then vice versa. There is just too many scenarios too consider for automagic 0% zero downtime switch over, what about instances conflicting in same app pool, what about locks on files (logs) plus a dozen other things that Im sure others will chine in on etc., only a simple site it might work. If you cant afford or justify the load balancer and multiple servers you need to determine if you as a software developer or infrastructure manager can accept from your client a 0% downtime SLA given your budget/resources, at this point you have to stand up and say "sorry, no money, no can do".



April 01, 2010 10:31
+1 Jason Dossett

I haven't watched the video but having the option to do deploy-time changes is a definite must IMO. You can't keep 'sensitive' information like connection strings in source control or in release configurations - the IT representative deploying the application needs a zero-low friction way of providing that information. Keep in mind that the deploy process should be viewed as an application where the deployer is the user.
Ambience
April 04, 2010 13:24

I doubt this will affect your scenario, if you truely do need zero downtime, then you need to load balance, when deploying, have the load balancer point at one server only until the other is up and then vice versa. There is just too many scenarios too consider for automagic 0% zero downtime switch over, what about instances conflicting in same app pool, what about locks on files (logs) plus a dozen other things that Im sure others will chine in on etc., only a simple site it might work. If you cant afford or justify the load balancer and multiple servers you need to determine if you as a software developer or infrastructure manager can accept from your client a 0% downtime SLA given your budget/resources, at this point you have to stand up and say "sorry, no money, no can do".


You're effectively telling me to say "no can do" to something that I have managed to do effectively and effortlessly on both Windows and Linux servers for the past four years. Granted, a load balancer would be sensible, but these decisions are sometimes more political than logical, and it is often not the developers/techies who have the final say in it.

Granted, there are cases where zero downtime is not going to be possible, for example, if you need extensive configuration changes, long running database change scripts, moving to a new server, and so on. But probably 80% of upgrades do not fall into that category, they are simple bug fixes or minor tweaks for which all you need is a new version of the code and in the worst case an iisreset.

What is more critical though is the ability to roll back in a single step. No matter how careful you are with your testing, there will be times when seemingly insignificant differences between your pre-production and live environments will cause your site to break, sometimes catastrophically. In this case, you will need to be able to roll back in seconds. I've found that with careful planning, this is possible even for complex deployments using the XCopy/IIS switchover method, and on one occasion (which was actually a particularly complex deployment) it allowed me to diagnose and fix the problem in only half an hour rather than spending an entire two days scratching my head and reporting the deployment as failed.
James McKay
April 06, 2010 23:14
Quick question... Why did the Web Deploy team not use T4 as the transformation markup. Just curious.

JR
https://www.google.com/accounts/o8/id?id=AItOawnLxxyPZSjwhbb8PlvsObBoW1MjHRrzjtQ
April 07, 2010 1:07
T4 is a generalized Text Transformation engine, but web.config files are XML, so they use a specialized XML transform technique. It's more terse.
Scott Hanselman
April 08, 2010 4:44
Thanks for the response.
https://www.google.com/accounts/o8/id?id=AItOawnLxxyPZSjwhbb8PlvsObBoW1MjHRrzjtQ
April 09, 2010 11:25
Hi Scott,

I attended your session about deployment at the Mix and wanted to try it out myself, for our own websites and development scenario. Now I discovered that the deployment features are not available for Web Site Projects.

I feel that the new deployment features in VS 2010 are a replacement for the VS 2008 Web Deployment Project (which Scott Guthrie mentions on his blog, http://weblogs.asp.net/scottgu/archive/2008/01/28/vs-2008-web-deployment-project-support-released.aspx). The Web Deployment Project supports Web Application Projects as well as Web Site Projects.

Why doesn't VS 2010 support both the project types, it seems like a missed opportunity? And will there be a VS 2010 Web Deployment Project in the near feature?
Martijn van Mechelen
April 27, 2010 15:06
Yes, I too wanted to express my big disappointment about web.config transformation not being available in web site projects,

It was really the single biggest thing to our project that we would like to use in 2010 (we are bound to 2.0/3.5 for at least another year). To read all this nice stuff about it, and either I missed the memo or not one Microsoft blogger clearly stated that it's not available for web site projects.

Really (really) disappointing.

Thumbs up for the feature, thumbs down for the communication about this issue (minor for most maybe, byt major for me).
Mike
April 28, 2010 8:37
Mike - Good news. You CAN use this deployment stuff with Web Site project. I'll do a post on it in a week or so, but the general idea is that since Web Sites aren't "compiled" you'll use a Web Deployment Project, just like in 2005/2008 and then the transforms will work during deployment or packaging. In the current
Visual Studio 2010 Web Deployment Projects Beta this happens at package time, but the word on the street is it'll be expanded by the RTM version soon.

You can also use:
msbuild mytest_deploy.wdproj /t:Package


Against your Web Deployment Projects to get your Web Site Projects to use transformations.

Help this helps and isn't "really really disappointing."
Scott Hanselman
April 30, 2010 3:15
Great video! It's finally all coming together. Thanks Scott.
http://jimmurray.myopenid.com/
May 06, 2010 12:41
Any idea on how to insert an element inside the configuration/runtime/assemblyBinding element? The problem I can't get around is that the assemblyBinding element has xmlns="urn:schemas-microsoft-com:asm.v1" and I can't get a xdt:Locator to find it or any sub elements.

Thanks in advance.
Terry Aney
May 13, 2010 1:11
Terry, we're looking at that, as we discussed in email, but sounds like you worked around it via:


 <runtime xdt:Transform="Replace">
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <probing privatePath="bin;Clients/_Assemblies"/>
      <dependentAssembly>
        <assemblyIdentity name="System.Web.Mvc" publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="1.0.0.0" newVersion="2.0.0.0" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
Scott Hanselman
June 01, 2010 0:14
I must admit I'm really frustrated. I've really tried using MSDeploy, MSBuild, TFS etc in a combination now, to get something sensible done in an enterprise scenario. However, The stuff just doesn't hang together at all.

If we take a look at Microsoft.Web.Publishing.targets, it looks like something thrown together a late night by a script kiddie. Lots of spelling errors, comments about TODOs because the IIS team does not support a feature yet, etc...

We have an enterprise application consisting of ~10 solutions, each with between 5-20 projects. I'm trying to use MSDeploy to package the stuff, for ~10 different configurations. But I'm just about to give up.

The documentation out there is dreadful, the blog posts are the best we get.

http://zvolkov.com/blog/post/2010/05/18/How-to-Publish-Web-Site-project-using-VS2010-and-MsBuild.aspx

was one of the best I've come over, combined with

http://msdn.microsoft.com/en-us/library/dd465318.aspx
http://vishaljoshi.blogspot.com/2009/02/web-packaging-creating-web-packages.html

etc.

It's just not good enough. I could have done this dozens of times easier in just a plain, old .NET language, but I so wish to do it within the build system that's provided for us...

Does anyone have any really good examples, books, advice, etc on this topic? Or is it just not feasible at all using MSBuild and MSDeploy?

Erik A. Brandstadmoen
June 03, 2010 11:22
Erik,
I understand that Microsoft.Web.Publishing.targets is a super long file and difficult to parse out... I acknowledge that we should do a better job in documenting it out, I will work on writing a blogpost on Web Publishing Pipeline (WPP) and various parameters that you can use from it... Hopefully that will ease out your pain a bit...
Thanks
Vishal
http://vishaljoshi.blogspot.com/
June 08, 2010 21:44
Is it possible to tie a build configuration to a branch in TFS? For instance, when you publish from the Staging branch, the Staging configuration is used (which, of course, uses web.staging.config). I'd have an easier time getting buy-in on the web config transforms if people weren't afraid of developers publishing without selecting the correct build configuration first.
This was the first thing I came back from MIX itching to implement, but I'm having trouble selling it for the stated reason.
Thanks!
Allen
June 15, 2010 19:08
Scott,

I usually don't take time to do this... busy like everybody else I guess. I just wanted to take a moment to thank you for putting this together in such a concise format and making it available to everybody. Your blog is a gold mine of tips and tricks for developers, and I really enjoy it very much.

Best wishes,

Eric Burcham
Eric Burcham
June 28, 2010 8:01
I still don't think this is something I will want to use with Team Build.

It seems transformations will require an entirely new build. Therefore, after I've deployed to dev, verified, and decided to promote the version to stage I will have to generate all new assemblies. How do I guarantee the two builds are consistent while time has most surely elapsed? Also, this generates an entirely new set of build metadata --labels, states, history, etc.

TFS Deployer seems to take a better approach. It hooks into build quality management allowing me to promote *the same* build from dev to stage, and eventually production all through a VSTS GUI (assistance is still needed from power shell). Of course, the long term desire is to have Microsoft streamline this process for us with no shell scripting required.
Karl
July 28, 2010 3:56
For those who might not have noticed, Vishal has a great tutorial here on using web.config transforms at the command-line for use in CI scenarios.
Scott Hanselman
August 07, 2010 8:25
I totally agree with Karl. If the answer to the CI Team Build scenario is to create a new build for each environment, it's going to be a show stopper. We would need to build the code ONCE, and decide at a later date to deploy to the development environment, smoke test, deploy the same build to the QA environment, thoroughly test it, and then deploy the same build to the production environment. I haven't been able to find an answer (yes OR no) if there is any way to support that scenario without having three CI builds just to support Web Deploy.

Other than that, it looks great.

RobG
https://www.google.com/accounts/o8/id?id=AItOawk5THNehDPImpyAUnLfY6suUt7zogtU6bA
August 09, 2010 8:10
Actually, this is very disappointing. So config transforms don't work while debugging (what if I need seperate settings from other developers?) and it doesn't work with app.configs? So, what's the point? Shorter syntax? So I guess I'm going to have to keep using the 'ole "CopyIfNewer.bat" eh? Seems like the extra 2 hours to get this feature working with the above scenarios would have been well worth it instead of me browsing the entire internet trying to figure out why my transforms weren't working...
https://me.yahoo.com/a/6mwLFxNiw9HXdwksmofryOmUF0EGVnST
August 25, 2010 17:51
From the other comments its sounds like it won't meet our needs. We have several customers on different servers with Test and Production instances. It would be great if the the XML transforms could be applied depending on where the web app is being deployed and not the build configuration.

As it is, we do a Deploy on a dev machine and zip up the folder leaving out the .confiig, customer specific menu xml files etc. When we deploy we unzip it over the old version and then hand edit the config etc. if necessary. Even if location specific transforms were not available it would be really handy to have a way of applying a config "delta", i.e. the changes to the config which are common to all deployments.

Does anyone know of a product that can do any of this?
Thanks,
Jason
JasonBSteele
August 31, 2010 20:19
Yeah, I agree that this is disappointing that it only works on deploy. (I also wasted a couple of hours trying to get this to work.) We have developers working remotely, so they all need different connection strings to debug, and resetting your connection strings every time you update your repo with everyone else's updates is a total PITA. Going back to build events and copyifnewer.bat
https://www.google.com/accounts/o8/id?id=AItOawmF-Z8BznMZJBteA-WCNWtMz9yh75hr8FQ
September 02, 2010 12:12
I was looking for this information since a long time.

Thanks for this Share.

Keep it up.
bulk sms indore
September 28, 2010 19:49
Hi,

If any of your readers would like to understand the XDT language better, I have written a very simple free implementation of it here.

http://petemontgomery.wordpress.com/2010/09/20/microsoft-xdt-language

Thanks! Pete.
Pete Montgomery

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.