Weekend mapping experiments, misc small but fulfilling development

This week I was working from home quite a lot, and given the holiday period, managed to get into the zone a fair bit.

It seems like this has continued into the open source work I'm doing. Most of the actual work is small at the end of the day, but has a larger impact.

So far; I've:

• Rewritten a few parsers for planning alerts, now that scraperwiki isn't working.
• Pondered sending in a pull request or two so that we can add Google Enhanced Search Results to OSM nodes via Schema.org/linkeddata.
• Effectively finished(!) an import of suburb boundaries in SA, so that the Adelaide Metro journey planner is more useful.
• Started a writeup of importing Geelong buildings, which are interestingly all mapped by the council with a LIDAR equipped plane.
• Done a proof of concept "detect a planning alert and trigger a note on OSM" script, which will evolve.
• Discover vespucci actually supports bing background imagery, allowing a bunch more approximate building traces to take place.
• Had most of the mapping between Gawler and Tanunda start to render (map below)
• Ordered some Sugru, to make custom fit headphones.

View Larger Map

Consuming GeoRSS and querying OpenStreetMap

In Adelaide, there was an unexpected fire. It was a bushfire, and I'd just been mapping the area earlier that day - ironically, tracing some of the forests which were being burnt out at that very point in time.

I decided to go and see what was available for fire alerts around the nation.

It turns out the majority of fire authorities publish GeoRSS. I took the feeds and aggregated them, plus put in a transformation of the data to JSON.

Using Leaflet, I added a rendering of particular fires.

Using the OpenStreetMap Overpass API, I queried for "all areas in a 500m bounding box", and again using Leaflet, rendered those polygons onto a map.

Here's the Project:
https://github.com/CloCkWeRX/burning-down

Here's a few screenshots:

Coverage, for most of Australia. WA doesn't provide openly licensed data, and South Australia's does not contain lat/long info.
The few you see in those states are from Sentinel, a satellite based system for detecting 'hot spots' that may indicate a fire.

Here, a number of resident buildings are marked as 'at risk' from a fire. In reality, these are probably safe, due to fires in these areas tending to be contained to residential areas.

On the other hand, you can certainly consider adding an Open Street Bug report if someone's house may have burned down!

This is more interesting - there's a fire or other emergency right next to a national park. National parks are made from trees, which in my experience tend to catch on a whole lot of fire!

Obviously this sort of incident is well in hand if the fire authorities are aware of it, but it is interesting to see that fuel sources are being picked up.

My next efforts here are going to be around encouraging edits near the area - if there isn't much detail on water sources (dams, swimming pools, rivers, lakes); it certainly makes sense to identify those if there's already been a fire once.

I also made a bit of effort to aggregate the feed output into sqlite - I might go run up a basic server and run this as a service at some later point, as aggregating this data seems useful.

The thing I like most about this - it's trivial to take any GeoRSS feed and query OpenStreetMap for nearby things - I've done fairly wide ranging queries with a narrow bounding box so far; but those are easy enough things to tweak.

Homeless data collectors

Here in SA, we have a container deposit scheme.
Homeless folk often are able to add a little revenue to their lives by collecting the 10c per can reward.

This benfits all involved, keeping areas clean, waste down and providing a limited sort of charity.

Given the low costs of digital cameras, gps, wifi and ardunio boards, is there a hidden opportunity for businesses?
For example, geocoding and address matching is an unsolved problem - we have data sets but they are not open, or can grow stale quickly.
Grabbing a photo of a location is possible from streetview, but again it is not an open data set.

Could rugged gps enabled cameras, with a 5c per photo reward, be a good solution?
Uploading and tagging against open street map, automatically?

Given a limited amount of review, this seems a feasible alternative to ocr solutions, with a real world benefit to people in need.

For the homeless involved, the fee is less than a can or bottle, time to take photo a bit more from location to location, but easier to carry.

Obviously there would be a discomfort factor for some home owners, but I would wager most folks already harvesting recycables deal with this already.

What other data could the homeless collect, given the right infrastructure?

Rosewater: 1; Car Thieves: 0

Where I live is often scorned at work - a bit of a working class area. To be honest, I never worried - I have been burgled in much nicer places than where I own my home.

So last night, when I was watching the Shield on TV I was a bit surprised to hear a sound like a handbrake being removed. Moments after, Christina starts calling my name. She rushes past me.
By this time, her car is ~10m further down the driveway than when we last left it. Christina leaps in, puts on the handbrake, and leaps out.

Not really knowing what to do, but by this time pumping all of my pent up rage and anger into a chest thumping display, I advance towards the car. I grab the keys out of the ignition and lock it.

Then I go out to the street.

All of this time, neither Christina or I have actually seen anyone.

It's at that point, a large black/dark new model 4WD puts on its lights. Enjoying the opportunity to have a yelling fit, I step out onto the road - barefoot, shirtless, true Rosewater uniform - and start having a focused chest thumping attack.
I look at the car - I can see the driver, dimly, but can't really tell if there's a passenger in there. I  step in front of the car, stick my arms out, and do my best frill neck lizard posturing.

They hesitate, at least just a moment. The angry cyclist in me starts to read the number plate, but before I can get it all they drive at me. Before I'm really certain what's going on, I've been pushed to one side and have gained an abrasion or two.

I get up, and turn after the car - now I have the fuckers! - and get an accurate read of the licence plate.

It gets a bit odd here. I run around like a headless chicken at one point or another, and the car locks/unlocks without me touching the remote keyless entry control. I think that's really weird that it's done that, and I start thinking Holden must use a RKE that is easily pwned?


The 4wd moves down the street, and I notice an individual - very tall, lanky, approx 50m away - standing or walked to the car. I can see they are tall because I can guesstimate their size against the Australia post box they are near.
I couldn't really say style of dress, but I got the impression they were young - 19-25. The individual is standing next  to the car.

Having seen them stop, both Christina & I consider the option they are coming back to assault us. Christina yells to get in the car, but instead I run inside. I find my trusty knife block, and pick a knife. I hesitate and swap to a cleaver. Bigger chopping = better chopping, don't you think?
I run back outside, ready to remove some limbs; but by this time they had gone - presumably further down the street, and onto grand junction rd.

Christina calls 000; and we're both quite annoyed with the operator - it takes ages to get an answer. When we are speaking to the operator, she's repeatedly telling us to calm down/slow down. This makes no sense to me at the time, as I feel lucid and clear. I tell her the licence plate ("STAIR52") and vehicle description; plus an assessment of myself (no ambulance, minor abrasions). She seems stuck on the assessing us, doesn't care about the crime.
I have to repeat the licence details repeatedly, then spell it out. It's like I'm calling car theft tech support. Eventually the operator checks and finds no plates that match the licence, I try one variation, but no luck.
She dispatches a patrol unit.

I check in with the neighbours - despite there being a party and other people awake just where the dark 4WD came from, no one has seen a thing. We have a chat and get to know the neighbours. Within 5 minutes or so, an ambulance officer is on scene - I  wave him down and give him a brief overview.
He looks, tells me "well, you are walking, so that's a reasonable sign. Enjoy the bruises!". We both agree there's no need to go to hospital.

We talk more the the neighbours - we find out the name of a dog who has been barking at us is "buddy" - and kill some more time. Eventually, about 30 minutes or more later, a patrol car arrives.
None of the details given to the 000 operator seem to have been conveyed to them. Theres "a new system" they are trying to make sense of.

They seem very disinterested. I can't really blame them, if I was a car thief I  would be 50% of the way home 30 min after it's gone wrong.
They look, and like Christina and I, can't see forced entry. This confuses them. Christina isn't very clear when they talk to her (we can't be certain if the keys were in the car or inside. We're confident now they were inside, but all I  saw was keys in the ignition).
The police grab my details, but when we try to tell them my mobile number isn't the greatest contact detail; they ignore it.
They do a cursory check of the car - no fingerprints obvious. They start to pack up and leave, and I have to interrupt them to tell them I saw the damned license plate, perhaps that would be relevant? I stated the plate number, gave descriptions etc.
They just left.
No report number or anything.

What pisses me off about the situation, apart from some jerks trying to run me over - the handover of details to the 000 operator by us was not a fluid process. The handover of information from 000 to police was also not a fluid process. Finally, the face to face interaction with the officers attending - they didn't care about the details.

Christina and I took some effort to write down what we saw at the time, and with 5 minutes of effort on a custom number plate designer, had pegged a likely match.

The fact it took 40+ minutes from start to finish makes me angry. Given the small population of Adelaide/SA (approx 1-2 million), if the details of the car had been put out within 5 minutes (assuming approx 60km/h + main road) - that's a 1 / 4 chance of catching them - Grand Junction, Torrens, Port Rd / Old Port Rd.
Within 10 minutes, it's about the same - a few more off roads (South Rd, Findon Rd, misc others).
Within 20 minutes though, you are in the CBD.
Within 30 minutes you are past the CBD and on your way to any other location of your choosing - or starting to calm down and find a back street or two to hide out in.

Right now, I'm a bit mixed about it - I can call / visit my local police station, but there's no email. There's @sapolnews and a few other Web 2.0 offerings which are a bit better; but I've got nothing there.

For me, I'm interested in the odds of putting this out to my social network - what can 150 people do to spot one car amongst ~1-2 million people given enough time and geographic spread?

Unlike some of the other times when I've been a victim of crime, I feel pretty relaxed - there's nothing more I could have done, bar picking up a brick or getting the cleaver first, to stop or apprehend the thieves.

I guess I'm kind of worried about the fact the car locked/unlocked without me touching the fob - I'm paranoid that RKE exploits are now in use and easily available. It seems the local police didn't credit it at all - they focused on trying to find fingerprints and signs of forced entry.
While it's likely that's the 90% of car thefts, there's a few papers findable via google since 2008 where weaknesses are demonstrated in modern RKE systems.
Also the notion of finding fingerprints seems almost laughable - you'd have to be pretty fucking stupid to avoid a pair of gloves - latex or otherwise.

All of today, both Christina and I spent looking at licence plates. We know it's highly unlikely that anyone locally did this, but we might as well keep our eyes peeled. We're interested to know if the car (STAIRS2) was stolen as well - it seems likely.
If it is stolen, it would be great to link the two incidents and pool information. Somehow, I'm not confident in the local police achieving that.

If you do spot someone driving a large, very new looking dark color 4WD around adelaide, with a licence of "STAIR?2", please do get in contact with myself or more importantly, the local police.

Transport

Boston's real-time transit data: "Better than winning the World Series" vs Adelaide.

Depressing.

I need a new housemate/tenant

Spice n Ice is probably the best indian restaurant I've ever eaten at.

It dislodged Raj on Taj, Indian Delights, and British India in a blaze of customer service; a camel masala special.

Why am I telling you about it? Because I need a new housemate, and if you think Rosewater/Port Adelaide doesn't have much to offer; Spice n Ice will utterly and entirely correct that misunderstanding for you.

Interested? Why not check out a place you can live within crawling distance of the best indian in South Australia.

Solar panels - how cost effective in South Australia?

I saw an advert from Origin Energy to install solar panels for $4990 (inc. GST and RECs rebate) for a 1.5kw system.

I have no idea if that's good or not, so started thinking about building a solar calculator. Turns out the Solar Shop is way ahead of me there.

Unfortunately their application is a little vague on the details, but suggests around 8-10 years is a typical time period to recoup your losses.

This letter appears to lay out details circa 2001 a bit better.

Savings and Loans Coast to Coast 09

The ride - it was pretty good. It was really easy compared to what I thought it would be, because it was all down hill pretty much.

I did 100km in 3 hours, 8 minutes (approx 31km/h average). Not many people passed me - the pro riders all started from Glenelg and did 120km; and passed me in dribs and drabs or as a whole team.

The rest were all intermediates like myself, and not many got past me at all. I spent a lot of time on my own, so if I was with a team or group, I could have made it even faster.

I'm not even too sore - just a little empty feeling.

When I finished I was miserable because it was cold, damp and I was dead feeling and alone - I didn't have anyone to share the moment with inside of 100km.

I've been overwhelmed by just how many people have messaged me and asked and stuff in the past two days - so yeah, its friends and family like Rob (especially Rob for training with me), Paul, Chloe, Rhianna, Laurie (and Kendy and Bob), Hitesh and Niall that have fixed that horrible dead alone feeling.

I don't really think I've ever known that before.

Its been an amazing two months of training for it and actually doing it; and boring the shit out of everyone with my incessant prattle.

Since this felt trivial, I'm probably going to try the other rides in the Grand Slam series - about 500km in all.

Followup

I still miss my old bike, but I worked out that it has cost me only $3.22 per km to buy the new bike; so far.

If this bike lasts me a mere 5000km (I've already done 500km on it since mid feb!) that'll be $0.36 per km I've paid for it.

Riding to work and back every day for a only year on this bike then throwing it out works out to a cost of $1 a day. Keeping it for 3 years - $0.33 or so.

Bus tickets would cost me at least $5.38 and be subject to price increases.

Savings and Loans Coast to Coast

Sunday, I'll be doing 100km or so in the Savings and Loans Coast to Coast 2009.

I opted for the cowardly start, but am probably fit enough to do the whole distance if I so choose.

Just tonight, I was happy to be able to ride for at least 5km holding my heart rate above 180 without discomfort; and still having energy left during the rest of the 50.

I felt bad having gone on a binge of eating, boozing and smoking on Sunday during the Norwood Food and Wine festival; and I certainly felt like rubbish for smoking tonight.

Let's see how I get on Sunday with the rest of the riders. I still can't keep fully up with the pack over just 50km, which is annoying, but I'm sure if I pace myself, I'll be just fine.

I've gone from

Date: 2009-01-06
Start Time: 06:15:00
End Time: 08:00:00
Time Taken: 01:45:00
Route: 2009-01-06 Route
Total Distance: 46.40 km.
Burned: 1,554 (kcal)
Workout Weight: 86.2 kg.
Effort: Normal
Quality: Good
Pace: 02:16 (avg) 
Speed: 26.52 (km/hr) (avg)

to

Date: 2009-02-24
Start Time: 18:30:00
End Time: 20:17:00
Time Taken: 01:42:00
Route: Outer Harbor group ride
Total Distance: 48.08 km.
Burned: 1,643 (kcal)
Workout Weight: 82.6 kg.
Effort: Easy
Quality: Good
HR: 150 (min) 164 (avg) 185 (max) 
Pace: 02:07 (avg) 
Speed: 28.28 (km/hr) (avg) 

in the last two months.

Some of that is new bike, some of that is losing several kg, but most of that feels like fitness.

Hurray.

Training for March

So I'm wanting to do the 120km 2009 Coast to Coast in March and I'm a bit worried.

So far, I'm not really doing anything too tricky. For instance, I just did a quickish ride - about 2-2.5 hours worth of effort there.


View Larger Map

That only came in at around 45km; it was in the evening, so not too hot, and I had to stop for a rest break or two on the Corkscrew road bit (take a look around, you can see what nearly killed me.)

At least when I took the short cut option home I got splattered with every insect known to man, and managed to see lightning as few have ever seen it - from right above Adelaide.

Update: How neat is this! A photo of one of the huger lightning strikes! This is the biggest one I saw, I'm pretty sure.



Two weeks ago, I did from my house to Mt Barker - about 4 hours in total one way (55km) with two generous rest breaks, then I kind of puttered along for a while.


View Larger Map

I got a lift from Hahndorf back to Crafers, cutting out 12-20 km of uphill on the return journey.

Total distance 89.8km - 12km of car ride.

Now I'm looking at this and thinking... whoa - 120km? Am I going to die doing it?

My next plans are to repeat the Mt Barker ride, but this time make it back.

The frustrating part of this all is that I can't seem to find anyone to partner up with - I tried getting a friend interested in getting fit for when he joins the police, but there was no chance. The only other people available are going to be Bicycle SA people; who will be fitter than me; or other people who are super super fit.
I either go too fast for my friends, or I drag down people who actually ride more than once in a blue moon.
Even worse, I'm thinking about buying knicks.

Letter to Kate Ellis

Dear Ms Ellis,
I am concerned about the ALP's policy to implement a Clean Feed internet filter. As I understand it, you are somewhat aware of this issue; and have met with Mark Newton; who discussed various technical details with you.

This is evidently an emotional issue from both sides – Sen. Conroy taking the save the children line, and many others, including myself, saying fine, but don't break my internet to do it.

My concerns stem from the fact I make my living as a software engineer, and my work literally could not exist without a free and open, responsive internet.
An internet filter as has been described would damage me; not to mention the Australian economy – it is no coincidence that breakthroughs in mass communication are tied to massive economic growth.

Also, by pursuing this technical solution so aggressively, Sen. Conroy has disregarded legitimate technical concerns; and demonstrated he is out of touch with the very field he is meant to be the Minister for.
I can only imagine that many employees of the Department of Broadband, Communications and the Digital Economy are embarrassed beyond belief; by the factual errors, by the rhetoric.
Let's not even touch on the fact they are meant to be providing both a Clean Feed (which will slow the Internet) and ubiquitous, fast broadband to all Australians.

So; my questions:

Is your position still one of support for the Clean Feed?
Have you researched more into the technical issues after your discussion with Mr Newton?
Would the ALP be open to different proposals; such as increased funding for policing and enforcement against child pornographers?
Or funding / encouraging the creation of ISPs like http://www.webshield.net.au/ ?


See also: protest pictures, nocleanfeed.

No Clean Feed Protests - Adelaide

I'm just up the street from the Adelaide No Clean Feed protest. It's pretty decent, and it's kind of mind numbing to be able to reach out and interact with the Melbourne protesters via SMS and internet.

We've got about 30-40; Melbourne has 50; and we've yet to hear in from Sydney.

There's shots of me even; how neat is that.

As soon as I can get home, I'll dump photos onto flickr.

Tune into the twitter #nocleanfeed stream.

This evening's ride

I'm working on riding my way up to Crafers from work - I got further than last time, but was still stuffed (I probably could have made it, but... willpower failed me)


View Larger Map

For those of you not from Adelaide, that whole Mount Osmond bit is pretty much... up.

Moving House on the Internet

I've recently decided to shift houses; and am simply amazed at just how easy it is now.

I found Share-Accomodation.net - just one of the many sites in this market space.

I really like the above - it's obviously thrown together by people without much English (typos abound); but the approach they have taken works like a charm.

You can post your ads and message for free - but you pay ($5 / 1 day membership or $20 / 1 month) to read your messages. By the time there's an inbox with several messages in it, it's much easier to justify spending the money.

It feels completely inoffensive, compared to the typical pay-before-you-can-message approach other sites take.

Highly recommended site

Death by Facebook

Every now and then the internet surprises you. I just found out a friend from high school is dead, by Facebook.

Since high school wasn't the most pleasant thing in the world to endure, I've dropped out of touch from most, so I didn't hear.

Goodbye Kane, I hope you had a good run.

Daylight savings considered evil

I don't really care when we do daylight savings in my day to day life. In my work life, however, I really do.

If it's not broken, don't mess with it.

Unfortunately, South Australia (where our servers are) decided to move the standard end date of daylight savings time by a week. Just for this year as far as I know.

This was not very well publicised here. Nor did it enable business to get their systems up to date.

This now means our server's PHP is broken, mysql is upset, PEAR::Date is upset and it has to stay that way for a week.

What a %#^#%ing mess.

This weekend's ride

My housemate and I went on yet another ride up impossible hills.


View Larger Map

Dear god, why do I do these things? I'm about dead now...

Adelaide E-Bus

Robby just let me know:

(2:00:45 PM) Robby: hey dan
(2:00:48 PM) Robby: http://www.ecogeek.org/content/view/1245/
(2:02:37 PM) Robby: if you look out the window
(2:02:39 PM) Robby: you can see the solar panels
(2:02:49 PM) Robby: i noticed them last week and thought it was just to power the new bus terminal

More feedback to the Government

I was wandering around the Green Vehicle Guide and I found a link or two about reusing their data - you just have to seek permission from the right people.

So I followed it and stumbled across this: Statement of IP Principles for Australian Government Agencies (read PDF).

It pissed me off to no end, as I've wasted time dealing with people who don't seem to be aware of this.

Basically; the document linked to boils down to:

• Open your data
• If a business a can use it; consider letting them
  
• More data, more access equals more jobs
• More access means the public is better informed
• Be flexible
  

The bit I really liked was:

Agencies should encourage public use and easy access to copyright material that
has been published for the purpose of:
.. snip ..
facilitating access to government services
.. /snip ..

So after being treated liked a moron the last time I tried to do something, the moment I saw a Feedback on Copyright / etc link; I bashed out:

This is just general feedback about public access to IP here in South Australia.

I recently contacted the DTEI here about gaining access to data - bus routes and timetables in a plain text format ('csv').

I wanted to create a 'google maps' type application to view the data on, and otherwise have better access to it. I had no commercial aims for it, and was very clear about that.

I also wanted it because the current way to view this data on the web is horrible and hard to use.

I'm a software engineer, so I [also] wanted to combine it with other data sets: for example a list of restaurants to create a 'find me a bus route to my restaurant booking @ 8pm Sunday'.

I was initially rebuffed; then after further persistence, I was invited a meeting - I thought I was in luck.

When I went in, after taking half of a day off from my full time work, I was completely stuffed around.
There appeared to be little or no understanding of 'Intellectual Property Principles for Australian Government agencies'.

I felt ridiculed by the way I was spoken to and treated.

Every question I asked was met with a look of confusion or a half hearted refusal.

While I can understand there is a balance between protecting commercial IP and public access, those I spoke to had no concept of a member of the public being able to make use of data.

I wasn't made aware of any other possible avenues or approaches I could take - basically; the matter was 'closed'.

I no longer wish to obtain this data, but I thought I'd let you know a first hand experience about us members of the public getting access to IP / data; and what (at least one) individual representing the government knows of your recommendations.

Your move, Australian government!


Update: My move again.

Dear Mr O'Connor

I refer to your email below to the Commonwealth Copyright Administration
requesting to use data - bus routes and timetables from The Department for
Transport, Energy and Infrastructure (DTEI) website. The DTEI administer
their own requests to use information from their website.

There is an online feedback form located at the following weblink
http://www.dtei.sa.gov.au/online_feedback_form.html

Regards
Commonwealth Copyright Administration
Attorney General's Department

Fined for Not Voting

Oh dear. I've been fined $280 for not voting in a state election. Evidentally, when I moved house, and they kept mailing the wrong address with warning notices, this escalated things.

They apparently took me to court.

The first I hear about it is a letter to renew my driver's licence sent to me; which has stamped on it the words "Criminal Law Sentencing Act", and "Magistrates Court".

That's somewhat frightening.

I rather thought no one actually bothered to enforce fines like that. I'm lucky, because the wonderful person at the Magistrates Court Fines unit / call centre was quick, helpful, and ever so friendly.

Lets just see if I can't get the same response from the people who are fining me...