Introducing NuGet Package Management for .NET - Another piece of the Web Stack
Microsoft's been filling out the Web Stack with more and more right-sized LEGO pieces lately, and today ScottGu announced the developer preview of NuGet. It's a piece of the stack that's been notably missing for years and after using it for a while now, I'm not sure how I lived without it.
NuGet is a package management system for .NET. The goal of NuGet is to make the process of incorporating third party libraries into your solutions as simple as possible.
Package Management itself is not a new concept. From Apt and "deity" before it at the system-level on *nix, to Ruby Gems, Maven, Synaptic, portage, dpkg, rpm and others, it's a well understood space. There are package managers for operating systems that install machine-wide libraries, and there are package managers for developers and their projects that manage dependencies and install libraries. NuGet takes inspiration from Ruby Gems and adds some .NET specifics.
NuGet - The Idea
Here's how it works. Notice the "Package Manager Console" window at the bottom of Visual Studio 2010. That's PowerShell. (It'll be in View | Other Windows for this release)
I type List-Package and NuGet goes to a central ATOM feed and retrieves a list of packages like this (not complete, I snipped it for the example)
Id Version Description
-- ------- -----------
Antlr 3.1.1 ANother Tool for Language Rec...
Artem.XmlProviders 2.5 Implementation of XML ASP.NET...
AutoMapper 1.1.0.118 A convention-based object-obj...
Castle.Components.Validator 1.1.0 The Validator component is us...
Castle.Core 1.2.0 Core of the castle project
Castle.DynamicProxy 2.2.0 Castle DynamicProxy is a libr...
Castle.Ioc 2.1.1 Castle Project offers two Inv...
EFCTP4 1.0 This CTP is a an early previe...
elmah 1.1 ELMAH (Error Logging Modules ...
tarantino-db-deployer 1.0.146.0 This is a database migration ...
WebFormsMVP 0.9.7.2 A simple Model-View-Presenter...
xunit 1.6.1 xUnit.net is a developer test...
At this point, as an example, perhaps I want to get the ELMAH (Error Logging Modules and Handlers) open source library setup in my MVC project. I have long said that ELMAH, as a project, deserves way more attention than it gets.
One of the reasons that it doesn't get used - as is the case with many .NET open source libraries - is that it takes too much effort to get it working. Unzip it, put it in a folder somewhere in your project, figure out what needs to be added to web.config, you know.
With NuGet, I type "Install-Package elmah" and it's done. If I wanted to be extra cool with PowerShell aliases I could have just typed "install-packageelmah."
PM> Install-Package elmah
Successfully added 'elmah 1.1' to MvcApplication4
And that's it. Elmah is automatically brought down to my machine, a reference is added to my project and everything it needs is merged non-destructively into my web.config.
I'll run my app and hit /elmah.axd just to prove it.
Complex Packages and their Dependencies
Looks good. But what if I want to add a more complex project, maybe NHibernate.Linq. From the Package Manager Console I'll start typing install-package nh
And the console reports:
PM> Install-Package NHibernate.Linq
'NHibernate.Core (>= 2.0)' not referenced. Retrieving dependency...Done 'log4net (= 1.2.1)' not referenced. Retrieving dependency...Done 'Iesi.Collections (= 1.0)' not referenced. Retrieving dependency...Done 'Antlr (>= 3.0)' not referenced. Retrieving dependency...Done Successfully added 'log4net 1.2.1' to MvcApplication4 Successfully added 'Iesi.Collections 1.0' to MvcApplication4 Successfully added 'Antlr 3.1.1' to MvcApplication4 Successfully added 'NHibernate.Core 2.1.2' to MvcApplication4 Successfully added 'NHibernate.Linq 1.0' to MvcApplication4
Notice that NHibernate.Linq knows its dependant on a bunch of other things. All those packages got pulled in and put in the packages folder. Notice the nupkg file there? That's just a ZIP file. Let's look inside.
Unzipping the nupkg we see lots of stuff. The binary is in there, but also a nuspec file that looks like this. Dig around in there and you'll find some standard packaging namespaces from openformats.org.
NHibernate.Linq
1.0
NHibernate Linq support is based on the existing, proven in production, Linq provider in NHibernate Contrib. The plans to overhaul that and merge that into NHibernate’s proper for the next release are still active, but the project team feels most strongly that production quality Linq support is something that we ought to provide for our users now.
Ayende
en-US
false
2010-10-03T16:32:29
2010-10-03T16:32:29
NuGet walks its way up the dependency chain and gets all the packages it needs, installing each. Packages can add scripts, content, references, and even run postscripts (ahem) written in PowerShell if they need something fancy.
I can even remove all that and its dependancies.
PM> Remove-Package NHibernate.Linq -RemoveDependencies
Let's add a few more interesting packages to my project. First, I'll add David Ebbo's wonderful T4MVC templates.
PM> install-packageT4MVC
Successfully added 'T4MVC 2.6.30' to MvcApplication4
David's project isn't actually DLLs, it's a T4 template and a config file, but no problem, it's added to the project.
Next I'll add the Entity Framework "Magic Unicorn" library that I love so much.
PM> install-packageEFCTP4
You are downloading EFCTP4 from Microsoft, the license agreement
to which is available at ....Check the package for additional dependencies,
which may come with their own license agreement(s).
Your use of the package and dependencies constitutes your acceptance
of their license agreements. If you do not accept the license agreement(s),
then delete the relevant components from your device.
Successfully added 'EFCTP4 1.0' to MvcApplication4
Note that this package includes some licensing stuff, as do many OSS projects, so I'm told as I get it. Finally I'll add SQL Compact Edition 4 as well with a little...
PM> install-packageSQLCE
Successfully added 'SQLCE 4.0.8402.1' to MvcApplication4
I've added NHibernate.Linq and four dependencies, then removed them all. I've added T4MVC, SQL Compact, Entity Framework CTP 4, and ELMAH.
We see we can add basically anything to a solution and it's only modifying the solution, and the packages folder next to it. Nothing is being added to the GAC (Global Assembly Cache) and no one is installing anything or messing with things system-wide. NuGet is about affecting local projects and solutions...and possibly adding functionality to enable me to make changes even faster...
Can you say Scaffolding?
I'll add one more interesting package. It's not a library like EFCTP4, or a database like SQLCE, or a T4 template like T4MVC. It's actually an example of PowerShell scripts that extend the Package Manager Console itself with new commands that interact with my project.
Note: This is just a sample/example, because we want to see and hear what you want, and how you want to use it. Talk is cheap, show me the code! ;)
Typing
PM> install-packageMvcScaffold
adds new commands to my console via a PowerShell script. It adds Add-MvcView, so I could so something basic like
PM> Add-MvcView Empty
Added file 'Views\Empty.aspx'
But that's not interesting. Let's assume I have a Code First class like this in my project and I've compiled once (in fact, a sample model is included with the EFCTP4 package):
using System.Data.Entity;
namespace MvcApplication4.Models {
public class Product {
public int ID { get; set; }
public string Name { get; set; }
public double Price { get; set; }
}
public class MyProductContext : DbContext {
public DbSetProducts { get; set; }
}
}
Since I've got a nice Product, why not:
PM> Scaffold-MvcViews -ModelType Product Added file 'Views\Product\List.aspx' Added file 'Views\Product\Details.aspx' Added file 'Views\Product\Edit.aspx' Added file 'Views\Product\Create.aspx' Added file 'Views\Product\Delete.aspx'
Which gives me a nice scaffolded set of views generated from T4 Code Templates (or your custom ones, if you like)
Pretty sweet. A lot has been done and prepped for me and all from the Package Manager Console within VS. I can personally see folks creating templates or meta-packages that encapsulate some CRUD (Create, Read, Update, Delete) best practices and sharing them via NuGet packages.
Can Haz UI?
I've focused on the Command Line, because it's awesome and "computers need to be hard©" but you could have certainly added your packages like this:
Clicking Add Package Reference brings up this dialog box. Look familiar?
From here I can search for packages, install, as well as see what packages I've already got installed. I can also check for updates and download new versions.
NuGet Developer Preview
Now this is just a Developer Preview, but look at it generically. We can interact with Visual Studio from PowerShell, package up Open Source libraries, scripts, content or perhaps even meta-packages (install-package hanselpack or install-package dearReaderLibs).
Phil talks about the Guiding Principles for NuGet on his blog. There will be a central feed with no central approval process for adding libraries. The community will police and moderate packages. But, anyone can host a feed. You can host an internal feed at your work, or even just point NuGet at a file system folder and share packages across your group.
Open Source
NuGet is interesting for a couple of reasons. NuGet isn't a classic Microsoft closed-source project; it's been accepted into the CodePlex Outercurve Foundation and is entirely hosted and managed in a Mercurial Repository at the CodePlex.com Forge. Developers from inside Microsoft and developers from outside Microsoft have been committing to the same repository. All the bugs and issues are managed transparently on that projects.
Yeah, we're slow, but we're working to turn this ship around. NuGet can, and has, accepted contributions, most recently from some folks on the "Nu" open source project. In fact, working with the Nu folks caused us to change the name of this project from its original name of "NPack" to NuGet. There's been a lot of "Source Opened" at Microsoft and a lot of "Open Source but No Takebacks" with some Open Source, but with NuGet I'm stoked to see us start giving (and taking back) in a more open way.
As Phil says, go over to the NuGet website on CodePlex and check out our guide to contributing to NuGet. It's early, and there's a LOT of work to be done, but we're planning to make this available for use in all editions of Visual Studio 2010.
Get the NuGet Preview today with ASP.NET MVC 3 Beta
Go get ASP.NET MVC 3 Beta and NuGet is included inside. Here's the direct download link.
To summarize:
- NuGet is open source and in the Outercurve Foundation
- NuGet integrates with VS and with PowerShell
- NuGet will have a community managed gallery without central approving authority
- NuGet won't mess up your computer or install anything outside your solution
- You can host your own feeds, or read packages out of a network share/folder
The whole team - inside and outside Microsoft - really hopes you like it.
Personal Aside: Changing Jobs
An a related aside, one of the reasons I came to Microsoft was to work on and encourage open source like NuGet. For the last few years I've been working in MSDN and STO (Server and Tools Online) most recently leading a community team with a great bunch of folks, including Joe Stagner, Jon Galloway, Rey Bango, Jesse Liberty, and Pete Brown. This week I'm leaving MSDN to go to work as Community Architect for the Web Platform and Tools (WPT) team under ScottGu, Bill Staples and Omar Khan. Pete will take over as the lead of my team and they'll all join Developer Guidance (née Patterns and Practices). We'll hang out a lot together, though, I have no doubt, and I'll be at the Patterns and Practices Symposium as well as PDC 2010 this year.
The WPT team includes ASP.NET, IIS, as well as open source projects like NuGet, ASP.NET MVC and Orchard. I'll be promoting Open Source inside and outside Microsoft, making sure the customers voice is heard as products are architected, as well as speaking and teaching whenever they'll let me. I won't be moving to Seattle. ;)
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
Does this work with nu gems? Is it supposed to supersede it?
Also, does it work without VS? Meaning, can I manage the packages in a command line?
Where I work, we use IVY dependency management for our backend (Java/Linux) system. This seems like it could be the piece we have been looking for on the Windows/.net side.
Blocking Issues:
This product requires Microsoft ASP.NET Web Pages 1.0. Please install the missing component, then try to install this product again.
Keep up the good work Scott and I'm looking forward to good things from you and your new team!
that was aimed at the same niche.
+1 this has been a sorely missing piece in .net for a good while.
I take it that this should also work for project types? (other than web)
Is there any harm in excluding .nupkg files from source control checkins? When are these package files used after an install?
Let's say I already have log4net in my solution and the package also needs it. Is it possible to tell the package to use existing reference as I do not want to have two copies of the same library or do I need to do it manually after referencing the package?
Ian - It borrows more from gems. Package Management like apt-get and deity before it in 1997 is a pretty well understood space. It's not rocket science. OpenWrap takes a different approach, hooking into MSBuild and creating references, much like Maven. There's also been Nu, Horn and other package management systems before, so there will always be familiar themes.
digiduck - The nupkg file has some metadata information in it that's not expanded, and they are used at uninstall time. File an issue, though at CodePlex, because you bring up an interesting post about file size.
Christof - Great question. We're working with Garrett. The basic idea is this. CoApp is system level package management ala apt-get. It's done at install time and affects the system, so if FireFox needs an SSL library at install time, it uses CoApp to get it. NuPack is developer-time and local to projects, like gems. So if I developer wants a library in their current project, they get it with NuPack. Make sense?
Bret - Yes, you're right! I missed that. This will be a nice way to update javascript libraries of all flavors.
Eduardo - It's entirely designed to impress the chicks. And maybe a few dudes.
Giorgi - Good question. I'll look into that. It's a valid use case we should track in Issues.
As a former one of those (rails nowadays, but still have Scott here in my reader list) I think it would a tiny but important point to perhaps give a bit more of 'hat-tip' to those that have traveled this path before. Package management comes from the various lively non-Microsoft, open source communities. Nothing in this is Nu is very new, and it would be at least polite to acknowledge that, i.e. Microsoft has not (wisely) invented this stuff so why not give credit where it's due at least in terms of history? Where's the harm in mentioning that in this blog post or in the project page (i.e. more)?
One of the reasons I think why non-Microsoft developers are so mean to Microsoft guys is that they give the impression of basically stealing and running with things that work, i.e. like a very slow burglar. While I know that not to be true, it does seem as if there are two worlds of which Microsoft is some shadowing mirror development shop which tends to follow on with in-house (always with the in-MShouse, sigh) clones about 3+ years after something is established.
Anyway, minor point but something to consider - standing on the shoulders of giants is fine, but don't forget to look down occasionally and say 'thanks'.
Also, as I'm thrashing around widely like a madman already, if Microsoft keeps sucking in and providing these projects centrally then don't be surprised when you find your community dependent on you and a little barren. Something it's good not to have a father-child relationship with your customer, but be a sibling instead (I say this as Scott, you'll be able to help with this as part of a new job perhaps). Encourage, grow and sponsor, but resist the urge to 'bundle it in-house'. The ones that say that MS developers 'prefer it all from one central place' are fools and should be ignored. :)
Cheers,
Ruben
Latest at my blog: And e/2 Appears from Nowhere! (follow up to 'And e Appears from Nowhere')
Might be an idea to work with the CodePlex.com guys to get an official NuPack CodePlex Repository that is built into NuPack by default and allows people with CodePlex Accounts to add new "gems".
Morgan - You can Create your own package here.
Just kidding. This is just simply awesome. I love that PowerShell is working its way into more an more areas. Give me the GUI for when I want to make things nice and easy to start off but PS when I want to get into the nitty gritty.
However, you all are making me work harder! 10 years ago Microsoft launched something new that was relevant to my work only like every few years. I had plenty of time to learn the dos and don'ts. Now it seems like every week there's some new freakin' cool technology that I have to use. Both a blessing and a curse. I have that trepidation that if I don't learn the new tools I'm not being as efficient as I could be. Definitely a bad feeling for a developer. Anyway, keep up the great work and I'll just have to dust off my GTD book.
Will it be possible for orgs to create their own internal NuPack repositories? I can imagine that this would be ideal for dev teams with complex interdependencies, and would be fantastic as a means of release management.
You can:
1. Just put them all in a shared folder structure and it'll be treated as a "feed"
2. Produce a feed and run a feed server/gallery (not created yet) on your intranet
3. Run an instance of the (not created yet) feed server/gallery.
Is this a restriction with the CTP version? If yes, will it be lifted in the beta?
Also, congrats on your new job. It looks cool to me to work with the Gu.
Hat tip for making it open source!
Now when will you have something like the rails migrations in .NET ? =)
Congrats for the new job. I wish you all the best, and hope this move will make the involvement of your division with the OSS community even deeper and better.
Then again, Apache loves their XML...
I do have to agree with brianary, though, the concept for NuPack sounds a lot like Perl's CPAN. The difference being that CPAN is normally machine-wide rather than just per-program/project/solution.
CPAN also occasionally suffered from dependency (or DLL) Hell, where certain packages would have dependencies on different versions of the same package, or a package version would be pulled, orphaning a package that depended on said version or newer.
Hopefully NuPack avoids this dependency Hell.
P.S. I heard from ScottGU's blog that a new beta of ASP.NET 3 and WebMatrix are out, including an updated version of Razor templates No love for them on your blog?
All I've seen so far are unverifiable feed URLs (e.g. http://173.203.67.148/feed -- no https means no server identity verification) with simple SHA checksums that are only good for ensuring that you downloaded the entire compromised packaged that an MITM attacker has prepared for you. ;-)
When I run the extension, the following error is shown:
"System.TypeLoadException: Could not load type 'System.Management.Automation.Runspaces.InitialSessionState' from assembly 'System.Management.Automation, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'.
at NuPackConsole.Host.PowerShell.Implementation.PowerShellHostProvider.CreateHost(IConsole console)
at NuPackConsole.Implementation.PowerConsole.HostInfo.get_WpfConsole()
at NuPackConsole.Implementation.PowerConsoleToolWindow.get_WpfConsole()System.InvalidOperationException: Can't start ConsoleDispatcher. Host is null.
at NuPackConsole.Implementation.Console.ConsoleDispatcher.Start()
at NuPackConsole.Implementation.PowerConsoleToolWindow.MoveFocus(FrameworkElement consolePane)"
I double checked and I have that assembly (same version and public token) in the GAC. Any idea of what that could be?
Thanks
www.padana.com
I didn't get one thing from it - is nap command an equivalent for Add-Package, it's just an alias or what?
I think NuGet is awsome, but I've hit a roadblock with it, and is that, since it's bounded to Visual Studio, if I'm behind a proxy (which I am) I cannot download the packages. Is there a tool to work this around or we'll just have to managa some kinky way? Btw, love ur blog...
Best regards,
David
Comments are closed.