NoSQL .NET Core development using an local Azure DocumentDB Emulator
I was hanging out with Miguel de Icaza in New York a few weeks ago and he was sharing with me his ongoing love affair with a NoSQL Database called Azure DocumentDB. I've looked at it a few times over the last year or so and though it was cool but I didn't feel like using it for a few reasons:
- Can't develop locally - I'm often in low-bandwidth or airplane situations
- No MongoDB support - I have existing apps written in Node that use Mongo
- No .NET Core support - I'm doing mostly cross-platform .NET Core apps
Miguel told me to take a closer look. Looks like things have changed! DocumentDB now has:
- Free local DocumentDB Emulator - I asked and this is the SAME code that runs in Azure with just changes like using the local file system for persistence, etc. It's an "emulator" but it's really the essential same core engine code. There is no cost and no sign in for the local DocumentDB emulator.
- MongoDB protocol support - This is amazing. I literally took an existing Node app, downloaded MongoChef and copied my collection over into Azure using a standard MongoDB connection string, then pointed my app at DocumentDB and it just worked. It's using DocumentDB for storage though, which gives me
- Better Latency
- Turnkey global geo-replication (like literally a few clicks)
- A performance SLA with <10ms read and <15ms write (Service Level Agreement)
- Metrics and Resource Management like every Azure Service
-
- DocumentDB .NET Core Preview SDK that has feature parity with the .NET Framework SDK.
There's also Node, .NET, Python, Java, and C++ SDKs for DocumentDB so it's nice for gaming on Unity, Web Apps, or any .NET App...including Xamarin mobile apps on iOS and Android which is why Miguel is so hype on it.
Azure DocumentDB Local Quick Start
I wanted to see how quickly I could get started. I spoke with the PM for the project on Azure Friday and downloaded and installed the local emulator. The lead on the project said it's Windows for now but they are looking for cross-platform solutions. After it was installed it popped up my web browser with a local web page - I wish more development tools would have such clean Quick Starts. There's also a nice quick start on using DocumentDB with ASP.NET MVC.
NOTE: This is a 0.1.0 release. Definitely Alpha level. For example, the sample included looks like it had the package name changed at some point so it didn't line up. I had to change "Microsoft.Azure.Documents.Client": "0.1.0" to "Microsoft.Azure.DocumentDB.Core": "0.1.0-preview" so a little attention to detail issue there. I believe the intent is for stuff to Just Work. ;)
The sample app is a pretty standard "ToDo" app:
The local Emulator also includes a web-based local Data Explorer:
A Todo Item is really just a POCO (Plain Old CLR Object) like this:
namespace todo.Models { using Newtonsoft.Json; public class Item { [JsonProperty(PropertyName = "id")] public string Id { get; set; } [JsonProperty(PropertyName = "name")] public string Name { get; set; } [JsonProperty(PropertyName = "description")] public string Description { get; set; } [JsonProperty(PropertyName = "isComplete")] public bool Completed { get; set; } } }
The MVC Controller in the sample uses an underlying repository pattern so the code is super simple at that layer - as an example:
[ActionName("Index")]
public async Task<IActionResult> Index()
{
var items = await DocumentDBRepository<Item>.GetItemsAsync(d => !d.Completed);
return View(items);
}
[HttpPost]
[ActionName("Create")]
[ValidateAntiForgeryToken]
public async Task<ActionResult> CreateAsync([Bind("Id,Name,Description,Completed")] Item item)
{
if (ModelState.IsValid)
{
await DocumentDBRepository<Item>.CreateItemAsync(item);
return RedirectToAction("Index");
}
return View(item);
}
The Repository itself that's abstracting away the complexities is itself not that complex. It's like 120 lines of code, and really more like 60 when you remove whitespace and curly braces. And half of that is just initialization and setup. It's also DocumentDBRepository<T> so it's a generic you can change to meet your tastes and use it however you'd like.
The only thing that stands out to me in this sample is the loop in GetItemsAsync that's hiding potential paging/chunking. It's nice you can pass in a predicate but I'll want to go and put in some paging logic for large collections.
public static async Task<T> GetItemAsync(string id) { try { Document document = await client.ReadDocumentAsync(UriFactory.CreateDocumentUri(DatabaseId, CollectionId, id)); return (T)(dynamic)document; } catch (DocumentClientException e) { if (e.StatusCode == System.Net.HttpStatusCode.NotFound){ return null; } else { throw; } } } public static async Task<IEnumerable<T>> GetItemsAsync(Expression<Func<T, bool>> predicate) { IDocumentQuery<T> query = client.CreateDocumentQuery<T>( UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId), new FeedOptions { MaxItemCount = -1 }) .Where(predicate) .AsDocumentQuery(); List<T> results = new List<T>(); while (query.HasMoreResults){ results.AddRange(await query.ExecuteNextAsync<T>()); } return results; } public static async Task<Document> CreateItemAsync(T item) { return await client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId), item); } public static async Task<Document> UpdateItemAsync(string id, T item) { return await client.ReplaceDocumentAsync(UriFactory.CreateDocumentUri(DatabaseId, CollectionId, id), item); } public static async Task DeleteItemAsync(string id) { await client.DeleteDocumentAsync(UriFactory.CreateDocumentUri(DatabaseId, CollectionId, id)); }
I'm going to keep playing with this but so far I'm pretty happy I can get this far while on an airplane. It's really easy (given I'm preferring NoSQL over SQL lately) to just through objects at it and store them.
In another post I'm going to look at RavenDB, another great NoSQL Document Database that works on .NET Core that s also Open Source.
Sponsor: Big thanks to Octopus Deploy! Do you deploy the same application multiple times for each of your end customers? The team at Octopus have taken the pain out of multi-tenant deployments. Check out their latest 3.4 release
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
Do you know if DocumentDB team is working on high level API?
If you will look on RavenDB please compare how hard it's to use DocumentDB compared to RavenDB.
For being the author of LiteDB I would be very honored :)
(small spelling in the last line "to just through objects at it and store them" => throw) Sorry, I had to ;)
Could you also share some experience of what that DocumentDB actually costs? The pricing model on Azure is pretty unclear for starters, 100 RU/sec => 5€ per month where 1 RU would be one document request with 1kb of data? Wtf does that actually mean?
1. In the real world you have to write retry logic to get around potentially hitting the RUs/second limit. This is super annoying, I just want to tell Azure, here is my credit card, take what you need. I don't want to have to write extra code.
2. If you have spikes in load, then you don't want to have to pay for RUs/second if you are not using them, so you have to write a web job to programmatically change the RUs/second to lower values when you are not using DocumentDB as much. DocuemntDB takes 10 mins to make this change, so it's not very reactive. Again, I'd much prefer if Azure could just work out what I've used and bill me for it, instead of my having to know up-front what I'm going to use.
3. In the real world you'd want to use partitioned collections so you can have more than 10GB of data stored. The minimum RU/s for partitioned collections are > 10.000 RU/s which is min $600 a month! So cost is prohibitive.
4. There are other limitations, many of which I see as 'now planned' which is good to see. See UserVoice https://feedback.azure.com/forums/263030-documentdb
DocumentDB is a very cool technology, a second generation
At Couchbase, we've also got a .NET Core library in developer preview (in addition to Java, Node, .NET etc). Check it out: http://blog.couchbase.com/2016/november/.net-core-with-visual-studio-code
They are pretty essential for any database, won't you agree?
thank you for the substantive feedback Muhammad, and we are making few improvements along those lines:
1)there are retry policies you can use in the SDK now. you don't have to write the code yourself
2) true. this is something we will be improving soon to handle spikes better.
3) you can now request a smaller partitioned collection via support (and soon via portal), with less RUs provisioned. the minimum cost is $150 now
4) ping me @kirillg_msft which ones are of particular interest to you, when you have a chance. Some of this "planned stuff" will appear within weeks. if you have a need for something like DocumentDB, would love to make it work for you.
That sounds great, I think it's only a matter of time (few months maybe) when DocumentDB will have all the required features that I'd want to use. It's maturing at a very fast rate and a review of it's features is a must when selecting a candidate for storing your data.
Please, that may be a change, but it's not nearly enough!
Make the darn thing local PERIOD!
Feel free to let me know if there is anything specific you are looking for.
Comments are closed.
Still, it is good to see content around the SQL vs NoSQL discussion. I for one as a strictly land-based code developer would appreciate more around this topic and it would be nice/valuable to see more diving around differences between the two outside what has already been discussed.