Scott Hanselman

NoSQL .NET Core development using an local Azure DocumentDB Emulator

December 04, 2016 Comment on this post [19] Posted in ASP.NET MVC | Azure
Sponsored By

I was hanging out with Miguel de Icaza in New York a few weeks ago and he was sharing with me his ongoing love affair with a NoSQL Database called Azure DocumentDB. I've looked at it a few times over the last year or so and though it was cool but I didn't feel like using it for a few reasons:

  • Can't develop locally - I'm often in low-bandwidth or airplane situations
  • No MongoDB support - I have existing apps written in Node that use Mongo
  • No .NET Core support - I'm doing mostly cross-platform .NET Core apps

Miguel told me to take a closer look. Looks like things have changed! DocumentDB now has:

  • Free local DocumentDB Emulator - I asked and this is the SAME code that runs in Azure with just changes like using the local file system for persistence, etc. It's an "emulator" but it's really the essential same core engine code. There is no cost and no sign in for the local DocumentDB emulator.
  • MongoDB protocol support - This is amazing. I literally took an existing Node app, downloaded MongoChef and copied my collection over into Azure using a standard MongoDB connection string, then pointed my app at DocumentDB and it just worked. It's using DocumentDB for storage though, which gives me
    • Better Latency
    • Turnkey global geo-replication (like literally a few clicks)
    • A performance SLA with <10ms read and <15ms write (Service Level Agreement)
    • Metrics and Resource Management like every Azure Service
  • DocumentDB .NET Core Preview SDK that has feature parity with the .NET Framework SDK.

There's also Node, .NET, Python, Java, and C++ SDKs for DocumentDB so it's nice for gaming on Unity, Web Apps, or any .NET App...including Xamarin mobile apps on iOS and Android which is why Miguel is so hype on it.

Azure DocumentDB Local Quick Start

I wanted to see how quickly I could get started. I spoke with the PM for the project on Azure Friday and downloaded and installed the local emulator. The lead on the project said it's Windows for now but they are looking for cross-platform solutions. After it was installed it popped up my web browser with a local web page - I wish more development tools would have such clean Quick Starts. There's also a nice quick start on using DocumentDB with ASP.NET MVC.

NOTE: This is a 0.1.0 release. Definitely Alpha level. For example, the sample included looks like it had the package name changed at some point so it didn't line up. I had to change "Microsoft.Azure.Documents.Client": "0.1.0" to "Microsoft.Azure.DocumentDB.Core": "0.1.0-preview" so a little attention to detail issue there. I believe the intent is for stuff to Just Work. ;)

Nice DocumentDB Quick Start

The sample app is a pretty standard "ToDo" app:

ASP.NET MVC ToDo App using Azure Document DB local emulator

The local Emulator also includes a web-based local Data Explorer:

image

A Todo Item is really just a POCO (Plain Old CLR Object) like this:

namespace todo.Models
{
    using Newtonsoft.Json;

    public class Item
    {
        [JsonProperty(PropertyName = "id")]
        public string Id { get; set; }

        [JsonProperty(PropertyName = "name")]
        public string Name { get; set; }

        [JsonProperty(PropertyName = "description")]
        public string Description { get; set; }

        [JsonProperty(PropertyName = "isComplete")]
        public bool Completed { get; set; }
    }
}

The MVC Controller in the sample uses an underlying repository pattern so the code is super simple at that layer - as an example:

[ActionName("Index")]
public async Task<IActionResult> Index()
{
var items = await DocumentDBRepository<Item>.GetItemsAsync(d => !d.Completed);
return View(items);
}

[HttpPost]
[ActionName("Create")]
[ValidateAntiForgeryToken]
public async Task<ActionResult> CreateAsync([Bind("Id,Name,Description,Completed")] Item item)
{
if (ModelState.IsValid)
{
await DocumentDBRepository<Item>.CreateItemAsync(item);
return RedirectToAction("Index");
}

return View(item);
}

The Repository itself that's abstracting away the complexities is itself not that complex. It's like 120 lines of code, and really more like 60 when you remove whitespace and curly braces. And half of that is just initialization and setup. It's also DocumentDBRepository<T> so it's a generic you can change to meet your tastes and use it however you'd like.

The only thing that stands out to me in this sample is the loop in GetItemsAsync that's hiding potential paging/chunking. It's nice you can pass in a predicate but I'll want to go and put in some paging logic for large collections.

public static async Task<T> GetItemAsync(string id)
{
    try
    {
        Document document = await client.ReadDocumentAsync(UriFactory.CreateDocumentUri(DatabaseId, CollectionId, id));
        return (T)(dynamic)document;
    }
    catch (DocumentClientException e)
    {
        if (e.StatusCode == System.Net.HttpStatusCode.NotFound){
            return null;
        }
        else {
            throw;
        }
    }
}

public static async Task<IEnumerable<T>> GetItemsAsync(Expression<Func<T, bool>> predicate)
{
    IDocumentQuery<T> query = client.CreateDocumentQuery<T>(
        UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId),
        new FeedOptions { MaxItemCount = -1 })
        .Where(predicate)
        .AsDocumentQuery();

    List<T> results = new List<T>();
    while (query.HasMoreResults){
        results.AddRange(await query.ExecuteNextAsync<T>());
    }

    return results;
}

public static async Task<Document> CreateItemAsync(T item)
{
    return await client.CreateDocumentAsync(UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId), item);
}

public static async Task<Document> UpdateItemAsync(string id, T item)
{
    return await client.ReplaceDocumentAsync(UriFactory.CreateDocumentUri(DatabaseId, CollectionId, id), item);
}

public static async Task DeleteItemAsync(string id)
{
    await client.DeleteDocumentAsync(UriFactory.CreateDocumentUri(DatabaseId, CollectionId, id));
}

I'm going to keep playing with this but so far I'm pretty happy I can get this far while on an airplane. It's really easy (given I'm preferring NoSQL over SQL lately) to just through objects at it and store them.

In another post I'm going to look at RavenDB, another great NoSQL Document Database that works on .NET Core that s also Open Source.


Sponsor: Big thanks to Octopus Deploy! Do you deploy the same application multiple times for each of your end customers? The team at Octopus have taken the pain out of multi-tenant deployments. Check out their latest 3.4 release

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
December 04, 2016 7:56
That POCO is hardly plain. It is tightly coupled with its serialization strategy. :P

Still, it is good to see content around the SQL vs NoSQL discussion. I for one as a strictly land-based code developer would appreciate more around this topic and it would be nice/valuable to see more diving around differences between the two outside what has already been discussed.
December 04, 2016 13:09
Hi Scott,
Do you know if DocumentDB team is working on high level API?

If you will look on RavenDB please compare how hard it's to use DocumentDB compared to RavenDB.
December 04, 2016 14:57
It's really interesting how hard Microsoft is trying to force people to use Azure for every simple things!
December 04, 2016 15:38
Hi Scott, when you write about RavenDB also comment about LiteDB, an open source C# NoSQL embedded database. Is single file as SQLite and has API based on MongoDB.

For being the author of LiteDB I would be very honored :)
December 04, 2016 17:50
Nice post Scott!
(small spelling in the last line "to just through objects at it and store them" => throw) Sorry, I had to ;)

Could you also share some experience of what that DocumentDB actually costs? The pricing model on Azure is pretty unclear for starters, 100 RU/sec => 5€ per month where 1 RU would be one document request with 1kb of data? Wtf does that actually mean?
December 05, 2016 12:45
I tried using DocumentDB in a production app a few months ago but had to pull back. A local DocumentDB instance was one of the blockers that's now gone but there are still issues:

1. In the real world you have to write retry logic to get around potentially hitting the RUs/second limit. This is super annoying, I just want to tell Azure, here is my credit card, take what you need. I don't want to have to write extra code.
2. If you have spikes in load, then you don't want to have to pay for RUs/second if you are not using them, so you have to write a web job to programmatically change the RUs/second to lower values when you are not using DocumentDB as much. DocuemntDB takes 10 mins to make this change, so it's not very reactive. Again, I'd much prefer if Azure could just work out what I've used and bill me for it, instead of my having to know up-front what I'm going to use.
3. In the real world you'd want to use partitioned collections so you can have more than 10GB of data stored. The minimum RU/s for partitioned collections are > 10.000 RU/s which is min $600 a month! So cost is prohibitive.
4. There are other limitations, many of which I see as 'now planned' which is good to see. See UserVoice https://feedback.azure.com/forums/263030-documentdb

DocumentDB is a very cool technology, a second generation
December 05, 2016 18:19
Hi Scoot, good post as ever,i dint know about that DocumentDB had a local emulator.... by the way, you know a way to connect to Oracle DB from .net core??
December 05, 2016 19:20
This would have been great about 2 years ago when I started using Document DB. I long since left though because there was no dev environment... oh well
December 05, 2016 19:25
Really cool to see some DocumentDB stuff in action on .NET Core. I really need to get around to checking it out when I get a chance.

At Couchbase, we've also got a .NET Core library in developer preview (in addition to Java, Node, .NET etc). Check it out: http://blog.couchbase.com/2016/november/.net-core-with-visual-studio-code
December 06, 2016 0:34
Nice that you mention a couple of things that are lacking but I did notice that you miss quite a few: https://feedback.azure.com/forums/263030-documentdb

They are pretty essential for any database, won't you agree?







December 06, 2016 2:23
Cool. This article scared me off NoSQL, but perhaps I should stop using that as an excuse and get amongst it.
December 06, 2016 8:39
Pretty useful information.
December 06, 2016 13:12
@Reztech: This is such a nonsense comment! You don't need to use DocumentDB if you don't like to use Azure. There are lots of alternatives. You cannot blame Microsoft for creating a product that is Azure-only. I love what they are doing with .NET core and ASP.NET Core. They are contributing a lot to different open source projects... What do they need to do for you guys to stop these nonsense comments?
December 06, 2016 23:19
@Muhammad Rehan Saeed,
thank you for the substantive feedback Muhammad, and we are making few improvements along those lines:
1)there are retry policies you can use in the SDK now. you don't have to write the code yourself
2) true. this is something we will be improving soon to handle spikes better.
3) you can now request a smaller partitioned collection via support (and soon via portal), with less RUs provisioned. the minimum cost is $150 now
4) ping me @kirillg_msft which ones are of particular interest to you, when you have a chance. Some of this "planned stuff" will appear within weeks. if you have a need for something like DocumentDB, would love to make it work for you.
December 08, 2016 18:51
About DocumentDB I like the server-side Javascript programming and the global distribution from Azure. Basically, this feature lets you scale your DocumentDB instance across different regions around the world and define what type of consistency you expect between the regions, from strong to eventual. It is even possible to configure an automatic and transparent failover over the different regions.
December 09, 2016 13:15
@kirill

That sounds great, I think it's only a matter of time (few months maybe) when DocumentDB will have all the required features that I'd want to use. It's maturing at a very fast rate and a review of it's features is a must when selecting a candidate for storing your data.
December 10, 2016 1:05
Emulator???

Please, that may be a change, but it's not nearly enough!

Make the darn thing local PERIOD!
December 12, 2016 4:46
@Slawek DocumentDB has several SDKs that you can use as high level APIs to interact with it. Details here: https://docs.microsoft.com/en-us/azure/documentdb/documentdb-sdk-dotnet

Feel free to let me know if there is anything specific you are looking for.
December 23, 2016 13:11
Hello! Excellent and decent post. I found this much informative, as to what I was exactly searching for. Thanks for such post and please keep on sharing with us.... Showbox for PC The website looks great. The article is written so professionally that no even words.

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.