Managing Multiple Configuration File Environments with Pre-Build Events
ScottGu mentioned an idea to me last week that he'd had for managing configuration files like web.config depending on what the current build config is. Bil Simser mentioned one part of this in January and Rob Chartier offered batch file help on a mailing list in June. Since ScottGu is busy Managing Generally (he IS the General Manager) so I said I'd prove the concept for him.
Here's the general idea. It's not too hard. I'll use an ASP.NET Web Site and web.config as an example, but this will work with most any kind of project, exe's or .dll's.
1. From Visual Studio, go File | New Project, and select ASP.NET Web Application.
Note: Do NOT "New Web Site" as we want a .csproj and we're going to use a Pre-Build Event, not supported by Web Sites. I've named mine FooFoo.
2. Right click in the Toolbars and ensure that the "Standard" toolbar is showing. You'll know if you see a dropdown that says "Debug" next to one that says "Any CPU."
Click the dropdown and select "Configuration Manager."
You'll probably have Debug and Release configurations, but you can also make custom ones and base them on existing configuration. In this dialog I've made a new "Deploy" and I'll base it on the "Release" configuration.
Make sure to create a Solution Configuration AND a Project Configuration, as they are different. Here I've made one called Deploy for the Project also. If you get an error message, be aware of the "Create new project configurations" checkbox. You might get a warning if you are making a new configuration and the dialog tries to make another configuration with the same name; uncheck the checkbox if that happens.
Of course, you can have as many Configurations as you'd like.
3. Add some custom configuration stuff in web.config, like connectionStrings:
<connectionStrings> <add name="Foo" connectionString="Data Source=localhost;Initial Catalog=DatabaseName; User Id=sa;Password=debug;" providerName="System.Data.SqlClient" /> </connectionStrings>
See now I've made the password in my nonsense connectionString = "debug"? Now, create three new web.config's by CTRL-dragging the web.config on top of the project. Name them web.config.debug, web.config.deploy, and web.config.release. Make the password equal to "deploy" and "release" respectively.
4. Ok, now we've got different configuration and different configuration files. Let's create a batch file called "copyifnewer.bat" and here's the contents:
@echo off echo Comparing two files: %1 with %2 if not exist %1 goto File1NotFound if not exist %2 goto File2NotFound fc %1 %2 if %ERRORLEVEL%==0 GOTO NoCopy echo Files are not the same. Copying %1 over %2 copy %1 %2 /y & goto END :NoCopy echo Files are the same. Did nothing goto END :File1NotFound echo %1 not found. goto END :File2NotFound copy %1 %2 /y goto END :END echo Done.
Basically this batch file will copy a file over another if the files don't match. It's not strictly "copyifnewer" (like, not at all) but it does the job.
Why bother with a batch file to check for changes and not just copy over the file every time? Well, each time you copy over a web.config it restarts all the designers and running AppDomains that are watching that file. No need to copy over a file if it hasn't changed...everything will churn less.
Put this copyifnewer.bat file in the root of your project.
Why not use PowerShell? One word - speed. Batch files are fast. Full stop. This is a build, so it needs to be fast.
5. Create a Pre-build Event. Right-click on your Project and select Properties. Click Build Events and in the "Pre-build event command line" and enter this value:
"$(ProjectDir)copyifnewer.bat" "$(ProjectDir)web.config.$(ConfigurationName)" "$(ProjectDir)web.config"
Notice the magic dust, the $(ConfigurationName) project variable, that contains "Debug" or "Release" or "Deploy."
6. Build. Now if you build, you'll see in the Build Output the batch file being run and the files being copied. Because it's a Pre-Build Event it'll be seen in both the Build Output in Visual Studio .NET.
When you build within Visual Studio the currently selected item in the drop-down list is the current configuration.
The configuration value can also be passed in on the command line when building with MSBUILD.
msbuild FooFoo.sln /p:Configuration=Deploy
And there you go. The connection string in the web.config now contains deployment-specific configuration data.
<connectionStrings configSource="separateConnStrings.config"/>
Bad things are that you've got to keep web.config's in sync if there's lots of settings, but you could totally break it apart via "include files."
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
The way we approach it here is to have "delta" files which specify how the config changes depending on the deployment environment. We originally started by using the Enterprise Library solution to that, but it's somewhat unstable (it has trouble with network drives, for some reason) and rather underspecified. As a consequence, we're now using our own powershell cmdlets to do the work.
The question of whether or not to do "whole file" or "delta" is a hard one. Your dev environment is likely to support and/or use many settings that are of no relevance in production. At least it will if, like me, you're in the habit of putting developer shortcuts into your applications. The real question comes in when you've got a new setting. Do you want that setting to automatically propagate through, potentially with the wrong value, or be ignored, potentially rendering your deployment dead?
I'd rather like Phil's idea of the "one true config file" to be generalized to the whole file, but then of course, you need a more complex syntax for expressing what gets overridden and when, which makes it harder to debug when things go wrong...
We have a build step that copies the relevant config file to match the build configuration or the machine. If a machine-specific config is present, it takes precedence over a build-configuration-specific one. Machine-specific-configs are rare.
LiveServerMachineName, LiveConnectionString, DevConnectionString
And have a "config" class with a shared (static) method who's call looks like:
config.CurrentConnectionString
So, the CurrentConnectionString method checks the current machine name, and if it's the same as the LiveServerMachineName, it returns the LiveConnectionString, else it returns the DevConnectionString.
The nice thing about this is, every developer gets the DevConnectionString while running on localhost, and when you deploy it you don't have to remember to do anything.
Of course, this only works in a simple "we have one live server" enviornment, but it is easily changed to work the other way (DevServerMachineName and reverse the IF) or to have multiple targets and options etc.
The point for me was to have ONE TRUE config file and not have to rely on someone remembering to switch files on deployment.
I'd like to have one master .config file, not 4 (one for each stage). Ideally I would like to create _override or xslt translation files to customize the master files for each stage. I'm just not sure how exactly...
Creating a specific build for each stage poses a risk, a small (and possibly manageable) one, but still... From a QA perspective I want to deploy only a tested and accepted build through to the next stage.
I love the way Telligent use _override files to modify custom .config files in CommunityServer (http://scottwater.com/blog/archive/cs-3-0-update/). This obviously doesn't work for Web.config's, nevertheless it's cool to be able to deploy a new version of CS without having to compare all tweaked configs.
In the machine.config for every computer (local dev workstations, stage servers, build servers, production servers), I add the following:
<appSettings><add key="Environment" value="Staging"/></appSettings>
Then, any configuration element that is environment-specific gets the environment appended, like so:
<connectionStrings><add name="Customers.Staging" provider="..." connectionString="..."/></connectionStrings>
<appSettings><add key="NTDomain.Staging" value="test.mydomain.com"/></appSettings>
Now, I can keep *all* settings in a single file. My apps take care of requesting the appropriate setting by looking appending the "environment" to the setting key.
Been doing this at clients since 2002 and it has worked well. To me, it's simpler than the post-build concept outlined above.
Caveats:
* If you don't want all settings in one file, this is bad. For example, if you use SQL auth on production, you will be sharing that password with developers. If you use Windows auth, it's not a problem.
* You'll want a small utility class to handle this. Mine is called EnvironmentalConfigurationManager and I'll post it online one day when I eventually get my garage sale up.
Just create one config file, and use specific tags for items you wish to vary. Then let your build server change these, depending on each build type.
For your developers, you will need to adjust their project file to do the same.
it's easy and very maintainable!
<configSections>
<sectiongroup name="Environments">
<section name="Local" ...></section>
<section name="Dev" ...></section>
<section name="Test" ...></section>
<section name="Prod" ...></section>
</sectiongroup>
</configSections>
<Local>
<add key="Description" value="My Computer" />
<add key="ServerName" value="MyComputerName" />
</Local>
<Dev>
<add key="Description" value="Development Server" />
<add key="ServerName" value="DevServer" />
</Dev>
Then in code:
Public Enum Options
Invalid
Local
Dev
Test
Stage
Prod
End Enum
Private Shared Sub SetEnvironment()
Dim env As String() = System.Enum.GetNames(GetType(Options))
For i As Integer = 1 To env.Length - 1 ' skip invalid environment
If LCase(HttpContext.Current.Server.MachineName) = LCase(ConfigurationManager.GetSection("Environments/" & env(i))("ServerName")) Then
_Environment = CType(i, Options)
_Description = ConfigurationManager.GetSection("Environments/" & env(i))("Description")
_Domain = ConfigurationManager.GetSection("Environments/" & env(i))("Domain")
End If
Next
End Sub
Check out the XmlMassUpdate task in the MSBuild Community Tasks project. It allows you to specify deltas within an XML file to apply to a different part of the XML file (or a different file).
So you could have something like:
<system.web>
<trace enabled="true" />
</system.web>
<substitutions>
<Debug>
<trace enabled="true" />
</Debug>
<Release>
<trace enabled="false" />
</Release>
</substitutions>
You could then call it in your BeforeCompile target, passing the $(Configuration) as part of the SubstitutionsRoot.
Another option is to use a dependency injeciton framework like StructureMap which has built in support for environment-based configuration.
"$(ProjectDir)copyifnewer.bat" "$(ProjectDir)app.config.$(ConfigurationName)" "$(ProjectDir)$(OutDir)$(TargetName).exe.config"
What I do is use nant to modify the web.config with deployment specific settings using the xmlpoke task.
Here is an example I found on Google.
Web Deployment projects will also do this for you, but unfortunately its looking like we will have to wait a bit before we can use those in VS 2008 though.
This works great until you let TFS control your projects. So, if you are using TFS, make sure to remove the web.config files from source control, but keep them as part of the project.
I use a similar technique, but I use a naming convention of "dev.web.config". That keeps the true file extension which ensures the proper security rules will be applied on the web server and helps with Intelisense when I need to edit these files. I store them in a solution folder and let the Team Build overwrite the primary web.config depending on the location the website is being copied to.
One more thing, most config sections allow the use of the "configSource" attribute. I keep most of the guts of the configs in separate files. That way, most of the environment-specific config files are very lightweight; just a bunch of references to mostly redundant config sections.
http://brennan.offwhite.net/blog/2007/04/26/post-build-deployments-with-msbuild/
The downside to this approach, the pre-processor directives usually end-up confusing the XML editor in VS; on the plus side, you have only a single file to keep up to date.
Why not use PowerShell? One word - speed. Batch files are fast. Full stop.
That's one for the ages.
Matt.
On a related but slightly different problem, I used msbuild to handle a different web.config file per developer. Documented here.
msbuild to the rescue.
Cheers
Matt
1. Have Build configurations for different environments. (DEV, QA, STAGE, PROD)
2. Create a custom section in web.config to hold environment specific settings and add/update code to access settings from this section.
3. Use an external config file to be configSource for this section by creating a env.config file. Additionally also create qa.env.config, stage.env.config and prod.env.config
4. Replacing the env.config with the correct external config source can be accomplished during build time or JIT compilation time. either way the effect is the same.
4a. If you just specify the configSource for custom section in web.config to point to an external file, these settings will picked up during JIT compile time.
4b. If you use Web Deployment Projects, "Enable Web.config file section replacement", the merge of external config will happen during build and packaging time, resulting in a web.config physical file with correct values.
5. Since most of use CC.NET and NAnt, have a step before the build to copy the correct env.config into the application directory, before proceeding with the build. Very similar to Project Pre-Build event.
Above will ensure that whatever build you are making will accommodate for the correct environment specific config sections to be available. Additionally we can have the different env specific external files to hosted in a Source control system and delegate the control over to suitable personal to manage their environments.
I'm in the process implementing this solution and would like to hear your opinion.
Thanks,
Prabhu
"The specified solution configuration "P3" is invalid. Please
specify a valid solution configuration using the Configuration and Platform
properties (e.g. MSBuild.exe Solution.sln /p:Configuration=Debug
/p:Platform="Any CPU") or leave those properties blank to use the default
solution configuration."
I can build it in VS without any problems..just command line that is causing me grief. Any ideas?
We control all of the app.config, etc files by doing something similar to Scott's examples, except we left out the "latest and greatest" check.
Good post Scott.
Comments are closed.
dodid something like this at Koders, except we use NAnt to copy over the correct file when running a build on the build server. For local machines, we'd do this manually. I like this approach. Nice and simple.Personally, I think trying to have as many settings in the database as possible is a good thing. That way, the differences between your various environments is minimal. Perhaps even just a connection string difference.
It'd be nice if you could deploy the very same web.config file to all your servers and each server would read the connection string that applies to it.
For example:
<ConnectionStrings>
<add name="Dev" connectionString="blah" />
<add name="Staging" connectionString="blah" />
<add name="Prod" connectionString="blah" />
</ConnectionString>
Perhaps the server would have an environment variable you set. Then again, you could just use configSource="localserver.config". I'm really just rambling here thinking out loud (or is it "out type"). I just want to avoid all that repetition.