LIXI forum 2010

Over the past two days, I have been lucky enough to attend the 2010 LIXI forum.

LIXI is a set of standards relating to how financial institutions can deal with loan applications, valuations, and a few other items.
Typically it's serialized into an XML version of a paper form; and gets sent around by SOAP to various workflow engines.

Previously, my involvement with LIXI has been in the valuations space, with ValEx being a big driver of LIXI usage; but no one really talking about the standards (or asking for improvements).

So, it was much to my surprise when I got to meet much of the industry over the last two days.

Probably one of the most interesting ideas put forward was by Christopher Joye. He proposed a national clearing house for data relating to credit:

Moving from the general to the specific, I would propose that the Commonwealth establish a central electronic ‘clearinghouse’ of all residential, personal and business credit originated in Australia. For simplicity’s sake, let’s call it the National Electronic Credit Register (NECR).

The main purpose of this information would be to provide much more accurate, real time, granular data to understand how individuals are actually fairing, rather than trying to guess at an aggregate level.
This is primarily to allow entities like the RBA to act quickly and accurately to help manage our economy.

I personally think it is a quite valid idea, as much of my frustration with how statistics and modeling takes place stems from the use of assumptions, or aggregate data.

There are obviously obstacles, not least of which is the potential for scope creep and how that relates to individual's privacy; but as a concept, it quite impressed me.

Other presenters touched on everything from the cost of software failure to a comparison of Australia's railway gauge mess to the current fragmentation of the LIXI standards.

The big move at the moment seems to be to standardize on a mechanism to transmit business rules.
Three vendors all have the same problem of trying to communicate changing credit policy for specific products from a lender to many brokers; and there are a number of solutions out there on the cusp of realization.

We saw practical demonstration of real time transmission of changed requirements (application A hits the save button, changing the age limit on a product, application B becomes aware of it within seconds).

Whilst people did not get into the technical details behind that particular demo, a technical training session on schematron was run.
I had initially dismissed the idea of schematron, but hearing Rick Jelliffe (one of the creators of XSD) talk about many of the concepts that drove the design; I'm more sold on it.

If you haven't used schematron, in short it differs from XSD by being much more relaxed. You write specific xpath rules to target segments of a document as being valid.
XSD on the other hand ends up being much more restrictive. For instance, it forces you to make choices about sequence of nodes and tends to fail validation if you have extra content.

The extra content item is a killer for the LIXI ecosystem, as everyone seems to have practice of grabbing a copy of the XSD at a given point in time; and if need be; doing minor modifications for their patch of it.
It's not on the web, because no one wants to expose a core banking system to the internet.
As an example, I introduced a simple additional Time element to ValEx's schema and the LIXI standards. It has taken months of testing with multiple organisations just to take a tiny step forward, and in the end a partner application still ended up ceasing to communicate with us.
The most painful part of it all? The only bits which "broke" in every single system were the XSD validation components. Every other bit of code simply ignored the additional XML.

With such a minor change requiring so much effort, you can understand how we've ended up in a fragmented state, and why XSD has been harmful.

I would be very keen to see the entire basis of validation (both at a syntax/expected data level and a business rules level) shift to schematron; or something else - much of the RDF world does not have the XSD problem, because no one uses XSD to validate!


Overall, I made quite a few contacts in all sorts of places, touching on topics such as the use of unique identifiers (ala Linked Data, even if we're not an RDF based standard); to improving physical documentation handling with our existing partners.

I look forward to the coming year, and the opportunities it may create.

On standards and marketing....

Through work we interact with quite a few other businesses. Some of them are in a similar space to us - providing tools for valuers to get the job done.

The way we all talk is a mixture of LIXI, an xml standard for describing valuations and other bits of the loan process, and SOAP to do the transport.

Modeling a valuation report is a very complicated thing - it usually takes at least two developers around 3 months to get the most common flavor 95 percent correct. Even then, because of the intricate variances; it takes a real valuer, not a developer, to work out the last 5%.

I know there are several valuation firms who deal with us who have undertaken this, and gotten it right. I've generally enjoyed working with them, because they just get it.

That being said, I still spend large parts of my day ever cautiously alert for yet another complaint about something not quite working - and it's largely attributable to one product in particular.

So; it comes as a huge surprise to find google adwords appearing on this very blog claiming blatant untruths about their product's fitness, features, stability and general standards adherence.

This product has cost me many hours of my working life, the company owner appears to be nothing more than a profit focused salesperson.

This is a constant source of frustration for myself and many of my coworkers.

In summary, I would advise anyone who is looking for valuation firm software / valuation management systems to strongly research their available options and choose wisely - it's a hard thing to get right, and some of the better marketed systems around are certainly not the right choice.

Making the papers again...

My workplaces made the industry news as a part of The value of valuation.

ValEx gets a mention with

While traditional valuations continue to play a large role in the loan application process, the valuation industry is undergoing some very significant technological changes that make the valuation process faster, much to the benefit of both brokers and borrowers.

Three big changes have altered the valuation landscape in the last few years: the establishment of LIXI standards, the introduction of exchanges and the growing acceptance of automatic valuation models (AVMs).

The introduction of web services technology that will enable lenders and valuers to exchange information in XML format has slowly been making inroads into the industry.

LIXI, the Lending Industry XML Initiative, has been actively promoting this method of communication. While valuation organisations seem keen to implement the standard, lenders have been slow to come on board, says Andrew Duerden, national sales and business manager at LoanWorks Technologies.

"There seems to be a bit of pull and push at the moment," he says, adding that legacy issues for lenders may be impeding their participation.

"They're running systems that are 10 if not 20 years old and to change this stuff to support new standards that are based on new technology is probably not a small task for them."

The second big change sweeping through the valuation industry is the establishment of valuation aggregators or exchanges.

Exchanges house a number of smaller valuation firms that have a broad geographical reach. Exchanges are advantageous to lenders in that they allow them to deal with one entity using the same automatic process.

The Valuation Exchange, which was formed in 2005, grew out of Megaw & Hogg National Valuers.

"We saw in Megaw & Hogg the need to change the way that valuations were being delivered to mortgage lenders, because mortgage lenders were looking for a much more streamlined approach," says Andrew Robertson, Valuation Exchange CEO. "So we saw [this] as an opportunity to provide valuations back to the lenders still from a wide variety of different valuation firms but on a consistent streamlined technology platform that would be much closer to the goal of straight through processing."

Individual valuation firms, particularly the larger, national companies such as WBP and Megaw & Hogg have spent considerable amounts of money on IT to speed up processing.

"We have to as an industry," says Megaw & Hogg's Guthleben. "We have to stay in touch with our client's requirements and change with them otherwise we become irrelevant to our client. So you need to adapt and continue to invest in IT and stay absolutely focused on quality and compliance - that's where my money's been going."

Sweet; job security!

Previously:
ANZ invests in ValEx.

xOperator, the Valuation Industry, XSLT, GRDDL and RDF

I just discovered the most amazing thing. It's a jabber/xmpp sparql interface, called xOperator.

It sits between you, the end user, and multiple other agents and datastores. It uses XMPP (Google talk!) to interact with you.

You say things like:

me: add ds DBPedia http://dbpedia.org/sparql

me: query SELECT ?name WHERE {
   ?musician skos:subject <http://dbpedia.org/resource/Category:Australian_musicians> .
   ?musician foaf:name ?name .
}

bot: (Alright, this is dummied up, but I *did* have it working with other stuff before)
"Ben Lee"@de
"Billy Thorpe"@de
"Brody Dalle"@de
"Charlie McMahon"@de
"Craig Nicholls"@de
"Daevid Allen"@de
"Delta Goodrem"@de
"Fritz Hart"@de
"James George Thirlwell"@de
"Joe Dolce"@de
"Johnny Diesel"@de
"Kasey Chambers"@de
"Kevin Johnson"@de
"Natalie Gauci"@de
"Percy Aldridge Grainger"@de
"Peter Garrett"@de
"Peter Allen"@de
"Rick Springfield"@de
"Richard Franklin"@de
"Sophie Monk"@de
"Steven Kilbey"@de
"Stevie Wright"@de
"Tina Arena"@de
"Toni Collette"@de

me: add template "Tell me about *" "PREFIX owl: <http: org="" 2002="" 07="">
PREFIX xsd: <http: org="" 2001="">
PREFIX rdfs: <http: org="" 2000="" 01="">
PREFIX rdf: <http: org="" 1999="" 02="">
PREFIX foaf: <http: com="" foaf="" 1="">
PREFIX dc: <http: org="" dc="" elements="" 1="">
PREFIX : <http: org="" resource="">
PREFIX dbpedia2: <http: org="" property="">
PREFIX dbpedia: <http: org="">
PREFIX skos: <http: org="" 2004="" 02="" skos="">


SELECT ?name WHERE {
   ?thing skos:subject <http: org="" resource=""> .
   ?thing foaf:name ?name .
}"

bot: template added

me: Tell me about German_musicians

me: Tell me about German_artists
bot: Store dbpedia answered:
     name
"A. R. Penck"@de
"Adolf Ziegler"@de
"Adolph Menzel"@de
"Albrecht Altdorfer"@de
"Alfred Harth"@de
"Alfred Rethel"@de
"Alfred Mahlau"@de
"André Butzer""@de
"Bruno Paul"@de
"Bärbel Bohley""@de
"Carl Grossberg"@de
"Carsten Höller""@de
"Charles Crodel"@de
"Christian Möller""@de
"Christoph Meckel"@de
"Christoph Ruckhäberle""@de
"Cosmas Damian Asam"@de
"Dieter Grossmann"@de
"Dieter Roth"@de
"Eberhard Bosslet"@de
"Eberhard Havekost"@de
"Egid Quirin Asam"@de
"Elsa von Freytag-Loringhoven"@de
"Ernst Deger"@de
"Hugo Höppener""@de
"Franz Joseph Spiegler"@de
"Franz Wilhelm Seiwert"@de
"Franz Xaver Feuchtmayer der Jüngere""@de
"Franz Xaver Feuchtmayer"@de
"Frieder Nake"@de
"Fritz Maurischat"@de
"Gerd Aretz"@de
"Gerd Arntz"@de
"Gert Heinrich Wollheim"@de
"Gert Neuhaus"@de
"Hans Haacke"@de
"Hannah Höch""@de
"Hans Grundig"@de
"Hans Richter"@de
"Heiko Daxl"@de
"Heinrich Aldegrever"@de
"Herma Auguste Wittstock"@de
"Herbert Holzing"@de
"Horst Janssen"@de
"Horst Antes"@de
"Ignaz Günther""@de
"Jankel Adler"@de
"Hans Arp"@de
"Jim Avignon"@de
"Johann Peter Melchior"@de
"Johann Baptist Straub"@de
"Johann Joachim Kändler""@de
"Johann Joseph Christian"@de
"Johann Melchior Dinglinger"@de
"Johann Baptist Zimmermann"@de
"Johann Michael d.J. Feuchtmayer"@de
"Johannes Baader"@de
"Jonas Burgert"@de
"Joseph Anton Feuchtmayer"@de
"Gotthard Johnny Friedlaender"@de
"Jonathan Meese"@de
"Joseph Beuys"@de
"Karin Ulrike Soika"@de
"Karl von Fischer"@de
"Karl Hubbuch"@de
"Klaus Herlitz"@de
"Konrad Peter Cornelius Klapheck"@de
"Kurt Wegner"@de
"Leo Kahn"@de
"Franz Karl Leopold von Klenze"@de
"Lore Lorentz"@de
"Ludwig Fahrenkrog"@de
"Martin Kippenberger"@de
"Manfred Mohr"@de
"Manfred W. Jürgens""@de
"Maria Innocentia Hummel"@de
"Mariele Neudecker"@de
"Werner Mathias Goeritz Brunner"@de
"Mary Bauermeister"@de
"Max Ernst"@de
"Michael Buthe"@de
"Matthäus Günther""@de
"Michael Sowa"@de
"Otto Eckmann"@de
"Otto Hupp"@de
"Ottmar Hörl""@de
"Otto Pankok"@de
"Otto Piene"@de
"Peter Vischer"@de
"Peter Dreher"@de
"Rebecca Horn"@de
"Ruben Talberg"@de
"Rosemarie Trockel"@de
"Sandra Rauch"@de
"Silvia Quandt"@de
"Thomas Eller"@de
"Thomas Schütte""@de
"Tilman Riemenschneider"@de
"Tomma Abts"@de
"Torsten Amft"@de
"Uwe Laysiepen"@de
"Veit Stoß""@de
"Virgil Solis"@de
"Wilfried Behre"@de
"Wilhelm Heine"@de
"Wilm Weppelmann"@de
"Wolfgang Joop"@de
"Wolfgang Müller""@de
"Wolfgang Tillmans"@de
"Evi Moechel"@de

(10:22:30 PM) clockwerx@livejournal.com/Home: Tell me about Australian_musicians
(10:22:32 PM) daniel.oconnor: Store dbpedia answered:
     name
"Ben Lee"@de
"Billy Thorpe"@de
"Brody Dalle"@de
"Charlie McMahon"@de
"Craig Nicholls"@de
"Daevid Allen"@de
"Delta Goodrem"@de
"Fritz Hart"@de
"James George Thirlwell"@de
"Joe Dolce"@de
"Johnny Diesel"@de
"Kasey Chambers"@de
"Kevin Johnson"@de
"Natalie Gauci"@de
"Percy Aldridge Grainger"@de
"Peter Garrett"@de
"Peter Allen"@de
"Rick Springfield"@de
"Richard Franklin"@de
"Sophie Monk"@de
"Steven Kilbey"@de
"Stevie Wright"@de
"Tina Arena"@de
"Toni Collette"@de

Big whoop you say.

Well, I've been working on both an XML_GRDDL package (php5), and the relevant GRDDL transformations to turn LIXI Valuation documents into the relevant RDF/XML representation.

I'm also re-deploying work's internal Jabber server in the not too distant future.

So:
1. We handle a pretty big chunk of all residential valuations within Australia.
2. A lot of that gets described adequately in LIXI
3. There's now or soon to be a GRDDL transformation or two for that
4. And a GRDDL parser for PHP
5. Add in xOperator
6. Add in robust jabber services
7. Mix in a few microformats as well

Viola:
My workplace is half a step away from becoming a knowledge enterprise as well as a valuation panel manager.

That's not bad for a few days off!

Imagine these future scenarios:

Working smarter with internal data and external clients:

Internal xOperator to answer questions like "show me similar valuations we've done to address xyz", and "show me everything valuer X has completed over $1million"

Paid commercial access to a SPARQL service as well, for larger clients who want to better understand the marketplace
Paid commercial access to an xOperator style service for smaller clients, who just want a couple of pre-built, on demand answers


Publishing old (> 12-24 months) LIXI data on the web
Merge our data up with other countries, like the US, for trend analysis
Provide anonymized public data, with http basic auth. to protect names, addresses, privacy
Merge diverse sets of data - ie, imagine EuroStat data for Australia, but with an atomic level of detail available about the raw input

Provide semi-automated valuations based on known data
Suburb X has a population of Y and an area of Z (db/wikipedia), factor that into calculations