How to Optimize Json.NET Serialization Performance

Newtonsoft is a pretty fast JSON serializer, but you can make it even faster!

By default, JsonConvert uses reflection to recursively search through the structure of an object during the serialization process. By implementing a custom JsonConverter that already knows the exact structure of the object, you can significantly increase serialization performance.

How much faster? That depends! The more complicated the data structure, the larger the performance gain. Below is a simple example...

Action	Method	Milliseconds	Performance Increase
Serialize	Standard	1134	115.59%
	Custom	526
Deserialize	Standard	1488	62.98%
	Custom	913

How to only Serialize Interface Properties with Json.NET

When you serialize an object with Newtonsoft's Json.NET it will resolve the serialization contract for the type being serialized. This means that if you want to serialize an object so that it matches one of the interfaces that it implements you will need to use a customized contract resolver.

When I first tried to do this I made a completely custom JsonConverter for the type that looked up the properties via reflection and just wrote their values out manually. Unfortunately had the side effect of bypassing all of the features the Newtonsoft provides with regard to decorating classes and customizing the serialization process for that object.

There was a good topic on Stack Overflow about this that led me to the custom contract resolver solution. However the sample implementation there is hard coded to only try to serialize one hard coded type for all serialization.

Below is an implementation (with tests) that allows you to specify a list of interfaces that you want to serialize by, and then if the object being serialized does implement that interface it will fall back on it's default contract.

InterfaceContractResolver Implementation

public class InterfaceContractResolver : DefaultContractResolver

{

    private readonly Type[] _interfaceTypes;

    private readonly ConcurrentDictionary<Type, Type> _typeToSerializeMap;

    public InterfaceContractResolver(params Type[] interfaceTypes)

    {

        _interfaceTypes = interfaceTypes;

        _typeToSerializeMap = new ConcurrentDictionary<Type, Type>();

    }

    protected override IList<JsonProperty> CreateProperties(

        Type type,

        MemberSerialization memberSerialization)

    {

        var typeToSerialize = _typeToSerializeMap.GetOrAdd(

            type,

            t => _interfaceTypes.FirstOrDefault(

                it => it.IsAssignableFrom(t)) ?? t);

        return base.CreateProperties(typeToSerialize, memberSerialization);

    }

}

Python 2.6 and HTTP Basic Authentication

I recently encountered an issue where adding basic authentication to some HTTP calls was breaking a Python application.

Come to find out there is a bug in Python 2.6 that appends a newline character to base 64 encoded strings. That newline character then causes your HTTP request to be malformed, so that the body does not match the content length. When consuming these malformed requests in an ASP.NET sever the body content would cut off early, and in the case of JSON content this made it so that the JSON string was incomplete and could not be parsed.

So what's the fix? You can either update Python, or fix your string after encoding.

Enjoy,
Tom

Deserialize Abstract Classes with Json.NET

Here is a fun problem: how do you deserialize an array of objects with different types, but all of which inherit from the same super class?

If you are using Newtonsoft's Json.NET, then this is actually rather easy to implement!

Example

Here are three classes...

public abstract class Pet { public string Name { get; set; } }

public class Dog : Pet { public string FavoriteToy { get; set; } }

public class Cat : Pet { public bool WantsToKillYou { get; set; } }

...here is an array with instances of those objects mixed together...

new Pet[]

{

    new Cat { Name = "Sql", WantsToKillYou = true },

    new Cat { Name = "Linq", WantsToKillYou = false },

    new Dog { Name = "Taboo", FavoriteToy = "Sql" }

}

...and now let's make it serialize and deseriailze! :)

Extending the JsonConverter

This tactic is actually quite simple! You need to extend a JsonConverter for your specific super class that is able to somehow uniquely identify each child class. In this example we look for a specific property that only exists on the child class, and Newtonsoft's JObjects and JTokens make this very easy to do!

Deserialize to ExpandoObject with Json.NET

I absolutely love Json.NET!

What I don't like is calling the non-generic DeserializeObject method and then having to deal with JToken wrappers. While these objects can be useful, I almost always want to just work directly with the data.

Good news, everyone! Newtonsoft natively supports deserializing to an ExpandoObject!

For anyone who does not know, ExpandoObjects are what .NET uses to let you create your own dynamic objects whose members can be dynamically added and removed at run time. The following two lines of code are ALL that you need to deserialize straight to an ExpandObjects:

var converter = new ExpandoObjectConverter();

dynamic obj = JsonConvert.DeserializeObject<ExpandoObject>(json, converter);

So why is this useful? To find out, let's take a look at some unit tests!

How JSONP Works

So recently I was having a conversation with someone about my JSONP update for the ExtJS library. We were talking about how I added error handling to their default implementation, and exactly what trick I had used to do that. However, we should probably start at the beginning...

What is JSONP, and how does it work?

JSONP is a standard (a hack really) that allows you to make AJAX requests across different domains. While this is an obvious security risk, there are also times where do right necessary.

Your browser appends a script block to your webpage that points to the foreign domain. Because of this, the JSONP request must be a simple get request that returns raw JavaScript.

So, what's the catch?

The return format for the JSONP request must be in the form of a call to a single global function. Once loaded, the script will execute and immediately call the global handler, which should know how to get the request data back to its caller. Also, depending on your browser, the script block may fire an onload (or in the hacky IE world, an onclick) event to help get the data back to its proper location.

The even trickier part is error handling, as there are no universal events to report script load failures.

Error Handling with JSONP

Ok, we are finally back on topic! As far as I know there are only three basic solutions to this problem...

Bad) No error handling at all.
Obviously this is a crappy solution, but it is what the default ExtJS implementation does. Somehow I suspect this lack of error handling support is 'justified' by saying that you shouldn't be using JSONP in the first place.

Better) Check for success after your max time out.
This is what my ExtJS implementation does. Before the request is made I set a timeout to call back 1 second after the max timeout of the request. The global handler calls a function that updates the JSONP request queue, canceling the timeout. If the time out is not canceled, it is assumed that the request failed and an error handler is called. This is much better than having now error handling, but it is still rather mediocre, as it could cause your end user to wait 30 seconds to find out about an error that happened 29 seconds ago.

Best) Wire up to state change events for the script block.
This is what the jQuery JSONP plugin does. Obviously this is the best solution, as it is using events exactly as they are supposed to be. The problem of course is that it has to support all the different browsers and all their different events.

In Summary

JSONP is useful, and is pretty easy to use. Implementing your own client side JSONP solution is kind of tricky, especially when taking error handling into account.

I have been trying to update my JSONP implementation to support additional error handling events, but so far it has proven to be rather difficult. Hopefully we'll see a 3.0 in the not too distant future, but right now I need to go watch the 2011 Womens World Cup Final! :D

~Tom

PS: Hopefully I will get the opporutnity to talk about this at SenchaCon 2011 in Austin TX...so if you haven't yet, go sign up!

Ext.ux.JSONP v2.0

The ExtJS library does include a JSONP component, but I found it to be lacking a very important piece of functionality.

The Problem: No Error Handler

Anyone who know's anything about JSONP is immediately going to call attention to the fact that JSONP isn't capable of supporting error handling by design. While that is true, it doesn't mean that we don't have to handle errors. One of the more well known jQuery plugins for JSONP includes a callback method for "complete" which enables you to know when a request failed.

One additional nitpick that I had with the ExtJS JSONP component what that I did not feel it was coded in a very Object Orient fashion, but that's probably just a personal gripe.

The Solution: v2.0

I rewrote the Ext.ux.JSONP component to include additional parameters that match up with a normal Ext.Ajax.request. This means that you can access handlers for success, failure, and callback.

Also, I made the code much more OO.

Download

Ext.ux.JSONP v2.0

Examples

// Old
Ext.ux.JSONP.request('http://api.flickr.com/services/feeds/photos_public.gne', {
   callbackKey: 'jsoncallback',
   params: {
       format: 'json',
       tags: Ext.fly('search-value').dom.value,
       tagmode: 'all',
       lang: 'en-us'                            
   },
   callback: updateResults
});

// New
Ext.ux.JSONP.request({
   url : 'http://api.flickr.com/services/feeds/photos_public.gne',
   params: {
       format: 'json',
       tags: Ext.fly('search-value').dom.value,
       tagmode: 'all',
       lang: 'en-us'                            
   },
   success : this.updateSuccess,
   failure : this.updateFailure,
   callback : this.updateCallback,
   scope : this
});

MVC JSON Model Binder Attribute

Here it is folks: the Ultimate Ultimate* ASP.NET MVC JSON Model Binder!

Moving complex data structures from client to server used to be difficult, but not anymore! Just add the JsonBinder attribute to your action parameters, and this Custom ModelBinder will automatically detect the type and parameter name, and deserialize your complex JSON object to the data structure of your choice.

No configuration required, it works every time, it's PFM!**

* Yes, I said Ultimate twice.
** Pure Friendly Magic

JsonBinderAttribute

public class JsonBinderAttribute : CustomModelBinderAttribute

{

    public override IModelBinder GetBinder()

    {

        return new JsonModelBinder();

    }

    public class JsonModelBinder : IModelBinder

    {

        public object BindModel(

            ControllerContext controllerContext, 

            ModelBindingContext bindingContext)

        {

            try

            {

                var json = controllerContext.HttpContext.Request

                           .Params[bindingContext.ModelName];

                if (String.IsNullOrWhitespace(json))

                    return null;

                // Swap this out with whichever Json deserializer you prefer.

                return Newtonsoft.Json.JsonConvert

                       .DeserializeObject(json, bindingContext.ModelType);

            }

            catch

            {

                return null;

            }

        }

    }

}

Controller Action

public class PersonController : Controller

{

    // Note: The JsonBinder attribute has been added to the person parameter.

    [HttpPost]

    public ActionResult Update([JsonBinder]Person person)

    {

        // Both the person and its internal pet object have been populated!

        ViewData["PetName"] = person.Pet.Name;

        return View();

    }

}

Sample Models

public class Person

{

    public string Name { get; set; }

    public int Age { get; set; }

    // Note: This property is not a primitive!

    public DomesticAnimal Pet { get; set; }

}

public class DomesticAnimal

{

    public string Name { get; set; }

    public int Age { get; set; }

    public string Species { get; set; }

}

jQuery Method

function sendToServer() {

    var person = {

        Name : 'Tom',

        Age : 27,

        Pet : {

            Name : 'Taboo',

            Age : 2,

            Species : 'Shiba Inu'

        }

    };

    // {"Name":"Tom","Age":27,Pet:{"Name":"Taboo",Age:2,Species:"Shiba Inu"}}

    var personJson = JSON.stringify(person);

    $.ajax({

        url: 'Person/Update', 

        type: 'POST',

        data: { 

            person: personJson 

        }

    });

}

Enjoy,
Tom