Scott Hanselman

NuGet Package of the Week: Microphone registers and discovers Web APIs and REST services with Consul

October 27, 2015 Comment on this post [9] Posted in ASP.NET Web API | NuGetPOW | Open Source
Sponsored By

I'm sitting on a plane on the way back from a lovely time in Europe. I attended and spoke at some great conferences and met some cool people - some of which you'll hear on the podcast soon. Anyway, one of the things that I heard mentioned by attendees more than once was the issue of (micro) service discovery for RESTful APIs. Now if you lived through the WS*.* years you'll perhaps feel a lot of this is familiar or repeated territory, but the new stuff definitely fits together more effortlessly than in the past.

Consul is a system that does service discovery, configuration management, and health checking for your services. You can write Web APIs in lots of things, Rails, Python, and ASP.NET with WebAPI or NancyFX.

Microphone is a library by Roger Johansson that plugs into both WebAPI and Nancy and very simply and easily registers your services with Consul. It's recently been expanded to support CoreOs-ETCD as well, so it's really a general purpose framework.

I made a little .NET 4.6 console app that self hosts a WebAPI like this.

namespace ConsulSelfHostedWebAPIService
{
class Program
{
static void Main(string[] args)
{
Cluster.Bootstrap(new WebApiProvider(), new ConsulProvider(), "HanselWebApiService", "v1");
Console.ReadLine();
}
}

public class DefaultController : ApiController
{
public string Get()
{
return "Hey it's my personal WebApi Service";
}
}
}

Now my Web API is registered with Consul, and now Consul itself is a RESTful Web API where I can hit http://localhost:8500/v1/agent/services and get a list of registered services. It's the Discovery Service.

Consul reporting my WebAPI

Then later in a client or perhaps another Web API, I can ask for it by name and I'll get back the address and port that it's on, then call it.

var instance = await Cluster.FindServiceInstanceAsync("Heycool");

return String.Format("Look there's a service at {0}:{1}", instance.Address, instance.Port);

Here's an active debug session showing the address and port in the instance:

Using Microphone.WebAPI and Consul for Service Discovery

It will be interesting to see what will happen with Consul and systems like it if the Azure Service Fabric gains traction. Service Fabric offers a lot more, but I wonder if there is a use case for both, with Service Fabric managing lifecycles and Consul doing discovery.

This is all early days, but it's interesting. What do you think about these new discovery services for Web APIs?


Sponsor: Big thanks to Infragistics for sponsoring the feed this week. Quickly & effortlessly create advanced, stylish, & high performing UIs for ASP.NET MVC with Ignite UI. Leverage the full power of Infragistics’ JavaScript-based jQuery/HTML5 control suite today.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

facebook twitter subscribe
About   Newsletter
Hosting By
Hosted in an Azure App Service
October 28, 2015 5:27
This is strong text. Very cool angularJS code in action.
October 28, 2015 6:11
This looks awesome but the docs imply quite a bit of work to set up a Consul cluster:

"A single server deployment is highly discouraged as data loss is inevitable in a failure scenario."

Would be great to get a Consul cluster option into the Azure Marketplace... anyone game? Hashicorp?
October 28, 2015 17:32
@Adrian Setting up a cluster looks pretty straightforward. They talk about using Vagrant in the docs (because they make Vagrant), but it looks like you would just install the package & run a commandline on 2 boxes, then a third to join them together. Later in the docs it shows that you can even tell one of the nodes to auto-join the cluster in the same commandline that starts it.

They highly discourage single server deployments because you'd be introducing a single point of failure to a network of microservices that will presumably come to depend on the service discovery that consul provides. Fault tolerance is the name of the game when you're spreading your business logic all over the place with microservices (for better or worse...) as networks will blip or partition, machines will reboot or crash, etc.
October 28, 2015 18:33
It's a shame that once again the wheel gets reinvented instead of leveraging the common udp discovery protocol

@scottt732 it would be much better if the service catalog was communicated peer to peer and as such need no central storage thereby removing all single points of failure. Of course this isn't possible without true service discovery or atleast a few nodes deemed superpeers that you can expect to be online to find atleast 1 other service.
October 28, 2015 19:15
In a world of many options sometime we just have to plumb for the simplest thing that'll work. I like this approach because it is just that. Simple. If there is enough traction in a product additional features (i.e. improved resilience) can be added. There has to be a kernel, a seed where things begin. Otherwise we'll all be sat around forever waiting for somebody to write the perfect solution.

As somebody you has starred at this problem before and blinked, I commend Roger for taking the time to write a solution, opensource it and then making an effort to evangelise it.

I'm sure Roger takes pull requests..?
October 29, 2015 2:00
Natural Zap - Immune Support - 25 Packet(s) All Pure Ayurvedic Herbal Supplement. Picture shown could differ in dimension, Herbal Zap Immune Help - 25 Packets.
October 29, 2015 17:18
I think you mixed up Roger's last name, it should be "Alsing", not "Johansson" ;-)

Nice read!
October 30, 2015 23:03
@ChrisMarisic Most systems that support clustering these days (Akka, Cassandra, Redis to name a few) use gossip protocols to exchange information about the nodes in the cluster. This is generally peer-to-peer. Typically no node is 'authoritative' about the topology of the cluster, but clients need to at least 1 IP address of an available node in the cluster cluster so they can discover other nodes in the cluster (it generally doesn't matter which node). If you pick only 1 IP and that machine isn't available, new nodes can't discover the other nodes on the cluster, which is why people typically define at least 2 known IP's/hosts. Akka.NET provides Lighthouse as a dedicated seed node, but even there their use is entirely optional. If you had a 4 node cluster and you knew that 4 would always be the smallest amount of nodes in the cluster, it would make sense to add all 4 IP's in the configuration for resiliency.
December 29, 2015 8:28
If you're using 4.6, then why are you still using string.Format :) ?
Av

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.