Flattening your models in JSON

You have a graph of objects, with some of those external lookup codes.

class ViciousAnimal < ActiveRecord::Base
  has_one :classification
end

# A simply code, description pair that captures the Smithson Nasty Bite Classification (WEAK_BITE, OW, WHERE_IS_MY_LEG)
class Classification < ActiveRecord::Base
end

When you are using something like knockout, quite often it's more useful for you to have application.classification.description present, but unless you do two serialisations to JSON, and build the relationship between the two JSON objects that's a bit annoying. In this scenario, I'll typically create an implentation like:

class ViciousAnimal < ActiveRecord::Base
  has_one :classification

  def classficiation_description
     classification.description
  end

  def to_json(options = {})
    super(options.merge({:methods => :classification_description}))
  end
end

This allows you to begin to flatten your object graph into something easier to work with on the UI.

What are the gotchas?

When you customise your to_json method, don't always expect rails to invoke it. For example, if you are passing back a number of objects, you'd typically do:

render :json => {
  :something => true,
  :vicious_animal => @vicious_animal
}

However, you'll be surprised to find none of your own implementation of to_json appears to be called. only whatever the Hash's to_json method supports.

To work around this, you often have to dumb your ActiveRecord instance down to a plain old hash:

render :json => {
  :something => true,
  :vicious_animal => ActiveSupport::JSON.decode(@vicious_animal.to_json)
}

Hardly the prettiest scenario.

How do you make that scale?

The above approach is only really good for a handful of descriptors - if you suddenly find half of your model is devoted to flattening out the graph, I would recommend extracting said code into a ViciousAnimalJSONSerializer, which knows how to orchestrate the mapping between the object graph and flattened view.

Rails provides ActiveModel, which gives you a good starting point for cleaner mapping code on plain objects. The benefit of getting yourself familiar with these sorts of mapping libraries is fairly high - for example, it would be fairly trivial to implement an n-triples mapper, to/from text serialization, or just about anything else you can imagine.

Knockout and metaprogramming

Knockout.js is great, but you often stumble as soon as you have a JSON object - how do you stuff it into the view model?

The manual touches on it a little, but understanding what the mapping plugin is doing is quite helpful too.

The main annoyance I've had with knockout is the accessors are all methods - I'm forever assigning to the view model object as though it was a hash or these were normally properties.

The trick to remember is you can treat an object like a hash, and iterate through all of it's members, invoking them.

var foo = {

    view_model: {
      application: function () {}
    },


    map: function (data) {
      var self = this, n;

      for (n in data) {
        if (!self.view_model.application[n]) {
          self.view_model.application[n] = ko.observable();
        }

        self.view_model.application[n](data[n]);
      }
    },
    init: function (application_data) {
      varself = this;

      self.map(application_data);

      ko.applyBindings(self.view_model);
   }
};

Obviously, this doesn't handle ObservableArrays and much more - for that I would strongly recommend you use the proper mapping plugin.

However, if you:

• Need to explore your view model
• Need to do light weight assignment of relatively flat data
• Or just want to get a feel for what's going on conceptually with mapping plugins

... then it's a handy technique to know.

Firebug, Prototype, Blogger & Json

Alright, I've had enough. Dealing with javascript is now too easy. Look:

<script src="javascript/prototype.js"></script>
<script src="javascript/scriptaculous.js"></script>
<script>
function render_headlines(data) { 
    var entries = $A(data.feed.entry);
    var ul = document.createElement('ul');

    entries.each(function(entry) { 
        var li = document.createElement('li');
        var a = document.createElement('a');
        a.setAttribute('href', entry.link.$t);
        a.appendChild(document.createTextNode(entry.title.$t));
        li.appendChild(a);
        ul.appendChild(li);
    });

    window.onload = function() {
        $("headlines").appendChild(ul);
    }
}
</script>
<script type="text/javascript" src="http://clockwerx.blogspot.com/feeds/posts/default?alt=json-in-script&callback=render_headlines"></script>

<div id="headlines" />

... which will get the ATOM feed of this blog as JSON, call render_headlines(); and append an ul to #headlines.

Thinking Out Loud - The Pain of Developing Bad Ideas

The problem with XUL & XMLHttpRequest is that XMLHttpRequest is not well suited to the transmission of server-side objects to the client-side ones. JPSpan fails miserably at what it does because of the lack of implicit sessions, the complexity involved, and the lack of queing and threading on the client side. This is no fault of the developers, they've done a great job at implementing a bad idea - and it seemed so right initially.

What we need is a language neutral intermediate data format. XML you say. Too easy. But the differences between Javascript and PHP clash far too much that though a PHP object goes into XML, a different, far more crippled object comes out in Javascript.

Both PHP and Javascript are all about doing one thing at a time. But Firefox has problems that interfere with the UI if you try to do synchronus XmlHTTPRequest connections, so you have to step into the world of Asynchronus connections. This is where the problems get worse. Callback functions. JPSpan relies on anonymous functions in callback handler varibles. Thus, you can't go: var mySale = new Sale(); alert(mySale.getTotal());. You have to rely on a function in two parts:

var handler = {
 getTotal: function(result) {
alert(result);
 }
}

function sell() {
var s = new Sale(handler);
s.getTotal();
//there is no way in this function to get the result,
//without resorting to global varibles.
}

That's incredibly more intricate than:

function sell() {
var s = new Sale(handler);
alert(s.getTotal());
}

Which is what we know and expect.

So, how do other AJAX applications manage broking objects where I'm failing?

Google's Gmail uses a UI engine which speaks javascript as the intermediate language. The server spits out javascript objects, there's no intermediate XML which makes us lose information in translation.

More 'traditional' web services (as this is all that we are really trying) do fine with RESTful interfaces and spitting XML at each other. There's a controlled and defined vocabulary, the server does their own thing, spits it out in hardcoded XML, and lets the client deal with the mess. This was the path I first started on, and I ditched it because when it got down to it, I was trying to pass strings about as my messages - messy, and I needed to pass more than one bit of information at a time.

PHP automagically understands GET and POST form submissions. XML via GET/POST as a restful service (ala musicbrainz) is an extra layer of abstraction ontop of that - but one we can deal with at the cost of losing our automagic varible parsing.

After writing this post, which was to entirely rethink the problem and forget that I'm screwing up quickly, I've realised that JPSpan isn't the way - the ideas are right, the implementation is too tangled. XUL, XMLHttpRequest, REST, XML. Well defined and controlled spec to start from. Evolves to meet real world implementations.

I recall initially I wanted RDF/XML over REST, but I abandoned this because RDF parsers in javascript where either closed to me (firefox security prevents me using the internal one) or not fully implemented (sorry Jim :/). How about RDF/XML over REST, parsed as XML by the client application, but more stable clients (like java ones!) could use RDF/XML to conduct transactions? I think I like that idea.

Next stop: Square One.