O’Reilly Author: Alasdair Allan – Astronomy and the iPhone

Where does Astronomy meet the iPhone? What changes when computing and connectivity become ubiquitous? Astronomer and O’Reilly Author Alasdair Allan is a fine example of the modern eclectic technologist. He is a hardware hacker, a software programmer and an expert in the arcane arts of iPhone App development. He is a Senior Research Fellow at the University of Exeter by day and by night he runs Babilim Light Industries. His first book, Learning iPhone Programming was published in March 2010, and the Rough Cut of his second book, iOS 4 Sensor Programming, is now available for download. He also recorded an O’Reilly Masterclass, Making use of iPhone and iPad Location Sensors. He is equally at home in his office crunching data or 15,000 feet up in the mountains on an observing run. He is at once a scientist of his day and a throwback to the generalists of the past. He blogs at the Daily Ack.

Your website says you are a “hacker, pretending to be an Astronomer, pretending to be a hacker” – could you explain what you do?

The way astronomers work is changing. Over the last few years isolated telescopes have been integrated into expanding smart telescope networks, spanning continents, and able to respond to transient events in seconds. At the same time the rise of data warehousing has made data mining more practical, and correlations between new and existing data can be drawn in real time. Astronomy, once a data-poor science, has become data-rich.

My own work over this period has focused on applying intelligent agent architectures and techniques to astronomy for telescope control and scheduling, and also for data mining. I’m currently leading the work at Exeter building a peer-to-peer distributed network of telescopes that, acting entirely autonomously, can reactively schedule observations of time-critical transient events in real-time. Notable successes include contributing to the detection of the most distant object yet discovered, a gamma-ray burster at a redshift of 8.2.

Fundamentally, I build the tools that let my colleagues do the best science possible. I’m one of those people who work in the cracks between disciplines.

How did you get to where you are? How did you get into technology?

I still consider myself a “working” astronomer, but I do very different work now than I did a few years ago. My background is observational astronomy, and my doctorate concerned multi-wavelength observations of Intermediate Polar systems. However as time has gone by I’ve found myself spending more time playing with the toys than doing science. I’m still publishing papers, but a lot of what I do now is focused firmly on the technology behind the science. I spend more time building software, and poking at hardware, than analysing scientific data.

My first real computer was a PDP-11, things pretty much were inevitable after that…

When did you encounter the PDP-11?

My father worked for a fabrication firm and they had a PDP-11, although I was never entirely clear why…

I didn’t run into better (more interesting) machines until I arrived at University and encountered a DEC MicroVAX and the University’s network of Sun SPARCstations. Although as a result of long term exposure to that network, years afterwards I still wince when anyone mentions thinnet and NFS mounted home directories in the same sentence.

Did you get to program it? Did you have to learn C in order to do so?

Yes and yes. I’m probably the last of the generation that had to self-teach themselves programming, or at least teach themselves computing without the aid of the Internet. I did have sporadic access to Fidonet, but back in the 80’s and early 90’s phone calls were expensive.

Bearing in mind the relationship between the iPhone and Objective-C, have you stuck to variants of C ever since?

No. Astronomers are generally Fortran people, so I spent many years as a Fortran programmer. But I also used C, C++, and Java fairly extensively, as well as several other languages . Later on I became a Perl guy, and until the arrival of the iPhone I spent most of my time writing in Perl. I love high-level languages, and Perl suits the way I think. Interestingly I know a lot of Perl people that have drifted towards developing for the iPhone and now really like Objective-C.

Follow Alasdair:
Learning iPhone Programming
Programming iPhone Sensors
The Daily ACK
Babilim Light Industries
Twitter, LinkedIn, Facebook, Google

What tools do you use in your work, both hardware and software?

For software problems I usually make use of Perl, for hardware problems I’ll reach for an Arduino board. For combined hardware and software problems I’ll probably also pull Processing out of the tool bag. I tend to use MySQL if I need to back my software with a database, although these days I also think about using Google’s App Engine to back my application in the cloud. I use Amazon’s S3 for storage and CloudFront for content delivery, and to a lesser extent their EC2 cloud for number crunching. I’d probably use EC2 more, but I’ve got a super computer in the building next door. That always helps…

What languages do you find most useful?

I try to be a language agnostic, all languages are useful, and not every problem requires a hammer. I try and pick my language to suit the problem, not force the problem to suit the language. That said, my “go to” language of choice to solve problems quickly is Perl.

How do you reconcile being a hacker and being an astronomer?

If you’ve ever visited a research laboratory, or a professional observatory, you’ll see there isn’t any need to reconcile the two. The Hollywood version of a laboratory is far too polished, you’re far more likely to see one-off pieces of hardware held together by spot welds, gaffer tape and luck. They probably look like most hacker spaces you’re familiar with except with institutional decorating. Most laboratory physicists are makers. Astronomers spend more time with computers than physicists, so most astronomers are hackers.

We spend as much time talking about computers over morning coffee as we do talking about astronomy.

What is it about the iPhone you like? Do you develop for any other mobile platform?

I used to develop for Series 60 devices, and PalmOS, and more recently I’ve played around with both Android and WebOS. However I think Apple have done an amazing job putting together a development environment that allows you to push their platform. I actually really like Objective-C, and the Cocoa touch frameworks are solid and well designed.

While I think Android in particular has a lot of potential, the development environment lags far behind Xcode and Interface Builder. Basically what I like about the iPhone is that despite being a theoretically “closed” platform it lets me scratch the itch far more easily than other devices. I can throw software together more quickly, and more efficiently, on the iPhone than I could on the other platforms.

If I could develop for Android using Xcode and Interface Builder I would. If anyone from Google is out there reading this I’d love to work with you to push Android development on the Mac forward using native tools.

What has the iPhone brought to Astronomy?

Look up into the audience from the speakers podium at most astronomy conferences and you’ll see a glowing sea of Apple logos staring back at you. Some time over the last decade, more or less corresponding with the arrival of Mac OS X, Apple has become the default computing platform for astronomers. We still have a lot of Linux servers, mostly hidden away in racks, but day-to-day, most astronomers have Apple Macbooks. The Apple Macbook, the iPhone, and now the iPad, have become tools-of-the-trade in Astronomy.

What has Astronomy brought to the iPhone?

Apart from me? We brought the Galaxy Zoo amongst other things, and you only have to take a look at the App Store to see how popular astronomy is on the iPhone and iPad.

The worlds of the iPhone and Astronomy seem very separate to the untrained observer – one is in your pocket, the other beyond the atmosphere. Is there a connection between the two?

A lot of people misunderstand my interest in the iOS platform. We’re moving away from general purpose computers towards more purpose specific devices. Among these are the iPhone and the iPad. The next generation of technology is ubiquitous computing, embedded devices, and mobile devices. Historically astronomers have always been on the bleeding edge of computing, and the processor inside my iPhone is more powerful than the processor inside the desktop machine I used to do my doctorate.

Why would an Open Hardware advocate such as yourself write a book about a closed platform like the iPhone?

I get asked that question a lot. Unlike a lot of purists I don’t view closed platforms as inherently evil. Technology is changing, we’re in the transition phase away from one model of computing and towards another. I’d argue that, despite the apparent closed nature of some of the devices, the underlying trend is towards more openness, not less. It’s just the things that are open and the things that are closed are changing, and with that the skills you need to work with the technology.

We’re moving away from general purpose computers towards more purpose specific devices. Amongst these are the iPhone and the iPad. The thing that’s changing here is that the tools and skills you need to play with the next generation of technology; ubiquitous computing, embedded devices, mobile devices. So you’ll need different skills than you might have used to play with the last generation of technology? The fact that you have to go out and learn some electronics to take part in the next big thing isn’t such a big deal.

What is it about the Arduino you find so appealing?

When I first started getting into electronics prototyping things in hardware was difficult. I ended up as a software person almost by default, it was just so much easier to try out ideas and see if they worked. However the current generation of micro-controller boards, especially the Arduino, has vastly simplified things. They’ve turned what used to be hardware problems into fairly simple software problems, and in the process speeded up hardware prototyping. Combining Arduino based hardware with Processing on the desktop gives me a great lever to attack interesting problems. Better yet, I’m now starting to use pre-release version of Processing for Android to quickly build Android applications to talk to Arduino-based hardware.

What happens when computing and Internet connectivity become ubiquitous?

Some people will become extremely rich, and some people will lose their shirts.

The signs of the next big thing; in the mainstream with devices like the iPhone, in academia with projects like Siftables and Google’s PowerMeter, and out on the open-hardware fringes with the Arduino, are everywhere. We’re entering a period of change, the next big thing is ubiquitous computing, and don’t let anyone tell you differently.

With open hardware like the Arduino arriving we’ve been given a new tool. In the same way the arrival of the desktop computer changed everything, the growing availability of open source hardware will do it again, and the death of the desktop computer might well spell the death of the Web, at least as we know it today.

In its place I think we’re looking at the arrival of something much more interesting, a pervasive information architecture. There is an important distinction to be drawn between simple embedded devices and ubiquitous computing, where you have a pervasive architecture of computing devices. Between independent gadgets responding to simple environmental conditions, and a pervasive information architecture shared across a number of ubiquitous computing devices.

Compared to a real ubiquitous computing we’re at the banging the rocks together stage:

“A big hello to all intelligent lifeforms everywhere…and to everyone else out there, the secret is to bang the rocks together, guys.” -Douglas Adams, The Hitchhiker’s Guide to the Galaxy

How can we use distributed data to make better real-time decisions?

In many cases it is possible to extend data warehousing to real-time assets such as geographically distributed sensor networks, and there are some interesting architectures that are emerging which present both static and real-time data resources using the same interfaces. They can be viewed as a general way to co-ordinate distributed (sensor) platforms, preserving inherent platform autonomy, using collective decision making to allocate resources. Interestingly some of these architectures that I’ve been working on can, and do, make increasing use of users as mechanical turks in their own software.

You’re building a near-space balloon – is that a work project or a personal project? Could you explain more?

It’s actually a work project supported by the University’s Annual Fund. Right now it consists of a bunch of parts spread across my desk, although hopefully we’ll get a couple of launches in before the British winter closes us down. I’m going to document the build as we go along and hopefully what will eventually come out of it is a generic kit that can go into schools, or enthusiastic amateurs can build. I’m eventually envisioning something a lot like Chris Anderson’s work on the BlimpDuino project, but for near-space (40km) weather balloons.

What is the maddest thing you’ve done for Astronomy – the thing you thought, ‘even I can’t believe I’m doing this?’

It’s a long list. Astronomers travel, a lot. I’ve stood on every continent except Antarctica, and there is astronomy going on even there. I work with some of the people doing that, so I’m still hopeful on that one. Most of the crazy happens when travelling, you end up in some of the remotest places on Earth.

Sometimes things get hectic, there was one time a couple of years or so ago when I flew out to a conference on the west coast, then back into the UK. I was in the country for just twenty hours, and then had to turn around again and fly back to the States for an observing run. The people paying for the conference, and the people paying for the observing run, were different. Neither of them were willing to pay for the one night of hotel accommodation in the States between the two trips, forcing me to fly home between the two. Let’s not get into the what that amount of jet lag feels like when you’re 15,000 ft up a mountain. On the bright side, I managed to see my wife for four hours, and do my laundry.

You do get relaxed about travelling, lost luggage, missed flights, interesting taxi rides. Although the time I got on the wrong plane on a dirt strip in the middle of the South African veldt, and ended up in Johannesburg rather than Cape Town, is still somewhat memorable.

Can you tell us a little bit about Babilim Light Industries?

My day job is as an academic, but I’ve got a lot of sideline projects going on. Enough projects in fact that I ended up rolling them up into a consulting business. I write software, for a time I was doing a lot of backend Perl for the banks, but these days it’s mostly for clients wanting applications on the iPhone. I build hardware, mostly distributed sensor systems; mesh-network based, usually using Ardunio and XBee as the basic platform. I also provide tailored training courses on how to build applications for the iPhone, I’ve done everything from single day introductory courses, to week long intensive deep dives, for various different clients.

Making use of iPhone and iPad Location Sensors

…and yes, I’m available.

How did you come to publish with O’Reilly?

I was in the right place at the right time. I was out at Apple’s WWDC in 2009 and my editor Brian Jepson, who I’d run into a few times at O’Reilly ETech and OSCON, tweeted that he was looking for authors with ideas for iPhone books. The rest is history: one book in press, another in Rough Cuts and an O’Reilly Masterclass under my belt.

My next book after the upcoming book on iOS Sensor Programming is probably going to be a departure for me, I’m going to hopefully sit down and write a book on “Making Citizen Science.”

Making Citizen Science sounds intriguing – explain more!

A lot of the commentators make the point that doing “real” science at home is pretty much impossible now due to the need for expensive equipment, and decades of experience. As a professional scientist I think that’s nonsense.

I think what people are missing is the shifting paradigm of how things are measured. I’m really excited by the possibilities suggested by (for instance) the next generation of smart phones, that hopefully might have many more sensors embedded (RFID, gas sensors, temperature/pressure) and for the large scale distributed sensor networks that might result. The web of things, ubiquitous computing and widespread availability of cheap hardware that makes good enough (not excellent, but good enough) measurements over a wide scale are going to give professional (and amateur) scientists a whole new lever on the world.

Amongst other emerging tools the Arduino platform has made building cheap sensor platforms within the easy reach of amateurs, and professionals, alike. One of the projects I’ll be talking about in the book is a pollution detecting hoodie which live logs geo-located level of pollutants to the Internet and alerts you to increased local pollution with embedded LEDs. You could imagine if they became widespread you could have a more advanced version that, either over mesh networking or perhaps Bluetooth, had peer-to-peer connectivity and stopped you going down streets that other people had already found to be too polluted.

Additionally some scientific disciplines (like astronomy oddly enough) have become data rich in the last few years, I literally have tapes and discs containing data I’ll likely never get round to analysing because I don’t have time. Likewise for all my colleagues, who have similar piles in their offices. The arrival of the new generation of all-sky telescopes (like the LSST) which will give us access to the sky in the time-domain isn’t really going to change that, it’s only going to get worse in fact.

All of these new trends puts doing “real” science at home well in the reach of most (educated) amateurs. Basically the push behind the book is measurement, you can’t have science without measurement. I’m hopefully going to try and teach people to quantise things, rather than relying on qualitative guesswork.

What does ACK in the Daily ACK refer to?

ACK, a packet message used in the Transmission Control Protocol to acknowledge receipt of a packet. I used to post a lot more frequently, so the joke was a bit more obvious, but blog posts have become somewhat sporadic over the last couple of years. These days I more frequently provides commentary in 140 characters or less.