Big data, climate change and developing economies: predictive modeling for improved lives

I did my first, and possibly last, Ignite talk at Strata London this week. If you don’t know the format it goes like this: 5 minutes, 20 slides; the slides automatically advance every 15 seconds and you have to tell your story and finish on time.

The story I was telling wasn’t about Acunu, nor even about low latency / real time analytics, which is a topic I’m often to be heard rabbiting on about. Low latency operational intelligence is what Acunu is all about. We build a platform for people who need to know what’s going on in their business right now. Pretty much every industry can benefit from tracking some KPIs minute-by-minute. And you can’t do that on day-old data in a data warehouse or Hadoop cluster, no matter how fast your BI tool can run queries.

But this time I was talking about a project dreamed up by some friends at Evidence for Development (EfD), a small charity I work with. Their idea is to build an economic model of the whole of rural southern Africa, including data about how people get access to food, market prices and meteorological data, then to build it up over a number of years until you have a database capable of predictive modeling of the effects of climate change in the region. The people at EfD have a ton of experience in building models of rural economies, which are widely, but not consistently, used in the region to help direct better aid projects and provide governments with early warning of disruptions (e.g. impact of crop failure).

There’s lots of support for the idea, but it’s going to be an interesting project to try to pull together all the strands. We’ve already made a start with getting the methodology onto the post-grad curriculum in economics departments in universities in the region, and starting to assemble open source developers to implement and support the software.

To see my talk you might have to refresh the page.

It was a challenge taking on the Ignite, lightning-talk format, but it was for a good cause and pretty enjoyable over all – apart for stumbling a few times as I raced to get through the story and keep in sync with the slides. I’d thoroughly recommend it to anyone who fancies themselves as a competent public speaker. I learned a bit that I hope will make me better in all formats and I am in awe of my fellow presenters.

Dai is a multi-decade veteran of the software industry, at some of the giants (Oracle & IBM) and some of the not-so-giant (Netezza and Acunu). He has worked in the trenches as an implementor, in the ivory tower as a text-book author and in a suit as a marketer. He loves learning about new technologies and how they can be usefully exploited – especially in ways that might make the world a better place.