On the nightmare that is JSON Dates. Plus, JSON.NET and ASP.NET Web API
Ints are easy. Strings are mostly easy. Dates? A nightmare. They always will be. There's different calendars, different formats. Did you know it's 2004 in the Ethiopian Calendar? Yakatit 26, 2004, in fact. I spoke to a German friend once about how much 9/11 affected me and he said, "yes, November 9th was an amazing day in Germany, also."
Dates are hard.
If I take a simple model:
public class Post
{
public int ID { get; set; }
[StringLength(60)][Required]
public string Title { get; set; }
[StringLength(500)]
[DataType(DataType.MultilineText)]
[AllowHtml]
public string Text { get; set; }
public DateTime PublishedAt { get; set; }
}
And I make a quick ASP.NET Web API controller from VS11 Beta (snipped some stuff for simplicity):
public class PostAPIController : ApiController
{
private BlogContext db = new BlogContext();
// GET /api/post
public IEnumerable<Post> Get()
{
return db.Posts.ToList();
}
// GET /api/post/5
public Post Get(int id)
{
return db.Posts.Where(p => p.ID == id).Single();
}
...snip...
}
And hit /api/post with this Knockout View Model and jQuery.
$(function () {
$("#getPosts").click(function () {
// We're using a Knockout model. This clears out the existing posts.
viewModel.posts([]);
$.get('/api/PostAPI', function (data) {
// Update the Knockout model (and thus the UI)
// with the posts received back
// from the Web API call.
viewModel.posts(data);
});
});
viewModel = {
posts: ko.observableArray([])
};
ko.applyBindings(viewModel);
});
And this super basic template:
<li class="comment">
<header>
<div class="info">
<strong><span data-bind="text: Title"></span></strong>
</div>
</header>
<div class="body">
<p data-bind="date: PublishedAt"></p>
<p data-bind="text: Text"></p>
</div>
</li>
I am saddened as the date binding doesn't work, because the date was serialized by default like this. Here's the JSON on the wire.
[{
"ID": 1,
"PublishedAt": "\/Date(1330848000000-0800)\/",
"Text": "Best blog post ever",
"Title": "Magical Title"
}, {
"ID": 2,
"PublishedAt": "\/Date(1320825600000-0800)\/",
"Text": "No, really",
"Title": "You rock"
}]
Eek! My eyes! That's milliseconds since the beginning of the Unix Epoch WITH a TimeZone. So, converting in PowerShell looks like:
PS C:\> (new-object DateTime(1970,1,1,0,0,0,0)).AddMilliseconds(1330848000000).AddHours(-8)
Sunday, March 04, 2012 12:00:00 AM
Yuck. Regardless, it doesn't bind with KnockoutJS either. I could add a bindingHandler for dates like this:
ko.bindingHandlers.date = {
init: function (element, valueAccessor, allBindingsAccessor, viewModel) {
var jsonDate = valueAccessor();
var value = new Date(parseInt(jsonDate.substr(6)));
var ret = value.getMonth() + 1 + "/" + value.getDate() + "/" + value.getFullYear();
element.innerHTML = ret;
},
update: function (element, valueAccessor, allBindingsAccessor, viewModel) {
}
};
That works, but it's horrible and I hate myself. It's lousy parsing and it doesn't even take the TimeZone into consideration. This is a silly format for a date to be in on the wire.
I was talking to some folks on Twitter in the last few days and said that all this is silly and JSON dates should be ISO 8601, and we should all move on. James Newton-King the author of JSON.NET answered by making ISO 8601 the default in his library. We on the web team will be including JSON.NET as the default JSON Serializer in Web API when it releases, so that'll be nice.
I mentioned this to Raffi from Twitter a few weeks back and he agreeds. He tweeted back to me
@shanselman if (when) we ship a v2 API, you can almost bet its going to be 8601 /cc @twitterAPI @johnsheehan
— Raffi Krikorian (@raffi) March 4, 2012
He also added "please don't do what the @twitterAPI does (ruby strings)." What does that look like? Well, see for yourself: https://www.twitter.com/statuses/public_timeline.json in a random public timeline tweet...snipped out the boring stuff...
{
"id_str": "176815815037952000",
"user": {
"id": 455349633,
...snip...
"time_zone": null
},
"id": 176815815037952000,
"created_at": "Mon Mar 05 23:45:50 +0000 2012"
}
Yes, so DON'T do it that way. Let's just do it the JavaScript 1.8.5/ECMASCript 5th way and stop talking about it. Here's Firefox, Chrome and IE.
We're going to do this by default in ASP.NET Web API when it releases. (We aren't doing this now in Beta) You can see how to swap out the serializer to JSON.NET on Henrik's blog. You can also check out the Thinktecture.Web.Http convenience methods that bundles some useful methods for ASP.NET Web API.
Today with the Beta, I just need to update my global.asax and swap out the JSON Formatter like this (see Henrik's blog for the full code):
// Create Json.Net formatter serializing DateTime using the ISO 8601 format
JsonSerializerSettings serializerSettings = new JsonSerializerSettings();
serializerSettings.Converters.Add(new IsoDateTimeConverter());
GlobalConfiguration.Configuration.Formatters[0] = new JsonNetFormatter(serializerSettings);
When we ship, none of this will be needed as it should be the default which is much nicer. JSON.NET will be the default serializer AND Web API will use ISO 8601 on the wire as the default date format for JSON APIs.
Hope this helps.
Sponsor: Big thanks to DevExpress for sponsoring this last week's feed. There is no better time to discover DevExpress. Visual Studio 11 beta is here and DevExpress tools are ready! Experience next generation tools, today.
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
new Date(parseInt(jsonDate.substr(6)));
Is much nicer, surprised that works... testing...
http://msdn.microsoft.com/en-us/library/bb299886.aspx
"March 6, 2012 12:15 AM"
UTC conversion issue? :P
"A value can be a string in double quotes, or a number, or true or false or null, or an object or an array. These structures can be nested." JSON.org
The latest version of Sugar handles the ugly date format from the default ASP.Net JSON serializer, but I really like using a standard date format, especially 8601. Hopefully all the JSON libraries will recognize this format.
I know you may hate the existing datetime format (I do to), but existing sites really rely on that, and this artical on MSDN Stand-Alone JSON Serialization.
Basically, I can't just update to use this instead of the existing api for a few reasons:
1) Breaking the contract - We have told our users that the date has to be in this format
2) We have a strict restriction on using 3rd party libraries, so now you are DEFAULTING to it?? Our legal department will have a fit!
3) This also means you won't be using the DataContractJsonSeralizer anymore? What about WCF? Is it also switching, or do we have issues within our own app (WCF expecting one format, and this giving another).
PLEASE DO NOT set JSON.NET as the default, but make it available, and fix the DataContractJsonSerializer to support more date formats than the \/Date(xxx)\/ one.
new Date(parseInt(jsonDate.substr(6)));
Don't do this, you are losing your timezone information.
Instead, just remove the \\ and / from the ASPNET string:
"\\/Date(1320825600000-0800)\\/".replace( /[\\/]/g, "")
This gives you:
Date(1320825600000-0800)
To convert this back into a JSON date, you can "new" the object in an eval:
var d = eval ("new " + "\\/Date(1320825600000-0800)\\/".replace( /[\\/]/g, ""))
Also, I just finally setup a W8 box and played a little with VS11 Beta and the WebAPI. The OS experience is a little disheartening yet, but I'm digging VS and WebAPI a lot... anxious to work with it more.
And then, what about, like us, a large corp that can't update everything at once, we have some using the old and some using the new format.
This means, even if we could get by our legal department (they look really hard at the license agreement) that we can NEVER use the new format, as it won't work with the systems that still require the old, unless they update DataContractJsonSerializer with support for more dates.
If you want to continue using DataContractJsonSerializer then why not create a MediaTypeFormatter that uses DataContractJsonSerializer? If Microsoft doesn't include one out of the box then I'm sure someone else will make one.
On licensing, Json.NET is the standard MIT license.
Also, Chad - we are changing only ASP.NET Web API...A product that isn't released. We aren't changing asp.net proper or WCF. Better? There is no breaking change as this is a new product, only now in beta.
All of your old code is most likely fine as it wasn't written to target the ApiController.
I don't see how this is a breaking change.
I don't think you read the post correctly. This change to the default serializer is happening in the ASP.NET Web API, not .NET. The standalone serializer isn't being removed. This change is only happening between the Web API beta and the final RTM. If your corporation is as large as you say it is, I seriously doubt you have production code deployed on a Web API beta that was only released a couple weeks ago.
TLDR: Nobody moved your cheese. Everything will be okay.
I was more making a point about the library (as we already use JSON.NET in some situations.
The problem I see now is one of incompatibility. If you aren't updating WCF, or ASP.NET, dates creating in one won't work in the other. So how do you suggest calling an existing service when you have a DateTime as a property in the serialized object? How do you know what format to put it in?
I know that JSON.NET can handle it, but it requires a special date handler, and DataContractJsonSerializer won't handle the new format. Can there at least be updates to make it EASILY work between services, without having to know what date format it is in.
DateTime.Parse handles several, but not the existing DataContractJsonSerailizer format, and then there is JavascriptSerializer - which is different.
I really like the simplicity of the DataContractJsonSerializer, and have heard several of my coworkers complain about JSON.NET.
In those cases, you can force JSON.NET to use the Microsoft form instead of the 8601 standard as pointed out by James in a comment above.
- JsonValueProvider for MVC model binding
- JsonResult from an MVC method
- Web API
- asmx services
- etc
Is there a voodoo initialize method(s) that says "just go do it for everything" I can call from Global.asax.cs?
im from south africa and we write our dates normally in the dd-mm-yy format. it makes sense, it goes from smallest, to biggest. i have never understood mm-dd-yy. please explain the logic behind it.. first list the medium one, then the small one, then the big one.
/weirdos
//your model
public DateTime PublishedAt { get; set; }
//new property
public string PublishedAtString { get{ return PublishedAt.ToShortDateString(); }}
This will display normal in view when passed with knockout and you won't have any problems passing it to controller on POST.
Hope it helps.
Thank for sharing this just in time!
Regards, always a pleasure to read your blog.
Thinking I was clever I spent quite some time using the Unix Epoch / 1970 / what is this madness??!! style dates in my own applications. In the end I came to the realisation that having something that can be read and understood by human eyes has real merit and so now I'm sticking with that.
Delighted to hear that this approach looks to be being adopted "officially". Marvellous!
Please make sure that it comes out with millisecond resolution and nothing more, that the time zone is always included. In cases of DateTime please use local machine timezone or UTC timezone of "Z".
This is how dates should be formatted from DateTime and DateTimeOffset.
"yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z'"
I can't tell you how much it pains me to see all these different so called ISO formats in the .NET framework that don't really reflect reality.
It would be great if Microsoft also supported this.
yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fffffffzzz (this is the "o" format, which is ISO 8601)
yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'fff'Z' (this is the industry standard ISO 8601 format)
It would save me a lot of time trying to create the most usable API across web clients, if Microsoft out of the box just supported this second format.
Thanks
I suffered from that nightmare, and decided finally to serialize in ISO 8601 as u did (Glad to do like U :)). but with cost!
Now if you tell me it's gonna be supported in future releases IT'LL BE GREAT!
I loved the "(new Date()).toJSON()" Part :)
Also, DataContractJsonSerializer is pretty bad with JSON Dates. It completely ignores the offset when deserializing, among other things. http://connect.microsoft.com/VisualStudio/feedback/details/723368
I'm super glad to head about JSON.Net being used for webapi instead. Question - is System.Json a different serializer than DCJS (in System.Runtime.Serialization) If not, why the new namespace?
The Json.NET change to use ISO 8061 is available from GitHub now. Right now it follows how .NET writes ISO dates in XML.
Great post, we actually bumped into similar issues not so long ago. As a consequence we actually coded our own JSON serailizer/deserializer a single and light yet powerful class named JsonUtilities which ends-up supporting three different date formats:
- \/Date(1330848000000-0800)\/, with or without “TZ“ (this is our default, and here's why: http://weblogs.asp.net/bleroy/archive/2008/01/18/dates-and-json.aspx)
- new Date(utc milliseconds)
- ISO 8601
Check-out this blog post if you want to know more: Dates and JSON
Cheers,
Carl
Are we going to see better odata support in Web API by the time it hits RTM especially for things like $inlinecount, etc that will make it possible to do paging and other stuff using it?
Thanks!
Post(string value1, string value2)
...this json.net formatter chokes on the json you send it (as opposed to when you just use the regular formatter) (say that json is {"value1":"one","value2":"two"} - the original formatter likes this, but the json.net formatter you use here doesn't). It gives this exception: {"ExceptionType":"Newtonsoft.Json.JsonSerializationException","Message":"Cannot deserialize JSON object into type 'System.String'. Line 1, position 12.","StackTrace":" at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader
Any ideas?
More annoying is getting back what you started with on the server, here's my solution to that problem. I'm sure it could be shorter.
Changing the way dates are serialized sounds like a recipe for disaster, though, I mean it's not really valid JSON any more.
public static DateTime FromJSDateTime(string jsDateTime)
{
Regex regex = new Regex(@"^""\\/Date\((?<ticks>-?[0-9]+)\)\\/""");
string ticks = regex.Match(jsDateTime).Groups["ticks"].Value;
DateTime dt = unixEpoch.AddMilliseconds(Convert.ToDouble(ticks));
return DateTime.SpecifyKind(dt, DateTimeKind.Utc).ToLocalTime();
}
And ultimately why worry about parsing dates at the client if all you were going to do was turn it into a string? C# is much better for formatting strings than javascript!
Adding Json.net will also help with other big issues like support untyped objects (type object), anonymous types, much better and two-way dictionary serialization, plus the ability to more easily plug in new converters if necessary in an easy way.
Adding Json.net is a great choice, but it's *stunning* to me that Microsoft couldn't get their own JSON serialization fixed with 3 different serialization tools (JavaScriptSerializer, DataContractJsonSerializer and now the new System.Json classes which are woefully incomplete and will compete with Json.net's feature set). Instead we'll have yet another DLL in the assembly soup that WebAPI adds to a project (can we get that consolidated please before RTM please???) and presumably in addition to System.Json which will add more confusion yet over what the right way to do low level JSON manipulation. <sigh> Two steps forward, one step back...
@Rob I also needed more control over JSON formatting, before I knew about Web API I created a JsonValueProviderFactory using JSON.NET. If you use my JsonValueProviderFactory and create a ActionResult that returns JsonConvert.SerializeObject(myModel); you will have completely swapped out the JavaScriptSerializer from MVC.
http://www.dalsoft.co.uk/blog/index.php/2012/01/10/asp-net-mvc-3-improved-jsonvalueproviderfactory-using-json-net/
JsConfig.DateHandler = JsonDateHandler.ISO8601;
One more pet peeve. Entity Framework returns DateTimes back as DateTimeKind.Unspecified. I save all of my DateTimes to the DB in UTC, but if the JSON serializer sees DateTimeKind.Unspecified, it figures it is local time and again, converts to UTC for a JSON Date(xxxxx) response. The result is double hour shifting!
I wrote a T4 template that forces all retrieved DateTime objects from the DB to come back as DateTimeKind.UTC by default, but a simple config file flag to force EF to return them UTC, built in, would be A-W-E-S-O-M-E.
My blog post on this: http://www.aaroncoleman.net/post/2011/06/16/Forcing-Entity-Framework-to-mark-DateTime-fields-at-UTC.aspx
--Aaron
There is nothing wrong with duck typing, but 'suck' typing (portmanteau of string and duck) is just asking for trouble. How do you fix it? The best I can come up with is:
{
"title": "test"
"posted": type.dateTime("2001-01-01T01:01:01Z")
}
This retains the 'jsony' paradigm where if you executed the string you would get the correct object (even though you shouldn't). 'type' would simply be an object passed through with a couple of methods (from what I can tell, only 'dateTime' would actually be needed).
var dataAUX = newcDate(parseInt(data.DataInicio.substr(6)));
var dataEND = dataAUX.getMonth() + 1 + "/" + dataAUX.getDate() + "/" + dataAUX.getFullYear() + " " + dataAUX.getHours() + ":" + dataAUX.getMinutes() + ":" + dataAUX.getSeconds();
This Way You Will get the hours as well
http://stackoverflow.com/questions/13886642/json-iso8601-parsing-in-javascript
Comments are closed.
Taken from my JSON prettifier: http://www.ajaxstack.com/jsonreport/