Later today I’m presenting at Where 2.0, talking about how cloud services improve data accessibility. The audience is government agencies and NGOs embracing the open data initiative, and us, the developers trying to work with the data. I will share specifics from our recent work trying to access large datasets (like SRTM elevation data, TIGER data from the US Census, and the OpenStreetMap Planet), and our positive and negative experience using Amazon cloud services to store and process this data.
Specifically, I’ll be talking about how we’ve downloaded and processed large datasets and hosted them in a cloud infrastructure for our own work and for the benefit of others. For developers, I hope that you’ll get a sense of what the actual important parts of the cloud are and how these play into our way of thinking about data and code. If you’re in charge of a large, underutilized data set, I’ll talk about experiences in accessibility and simple changes that can make data more practically available and useful. Even if you’re new to the area, this will provide a good background of the challenges we’re facing and the landscape of computing on the web.
Background
Some government data has long been publicly available, but only available as zipped files on unreliable FTP servers. NGOs have run into similar issues with vital data licensed for free use – but locked up in PDF files. What does this mean? That the data was free but underutilized, and a cottage industry of for-profit CD-makers and data compilers sprung up around the issue. Cloud hosting offers agencies a better way to make their data available, at lower cost and orders of magnitude faster than downloads. From a cloud computing server, you can now instantiate and connect to data within minutes and perform analysis, visualization, and extraction immediately. You can then take the results and redistribute them with cloud-computing power or protect them behind a firewall. Bottom line, it makes data more usable, useful, and used.
Presentation
Here are my slides if you want to skim them a head of time or can’t make it.
My presentation is at 5:25 pm today (Wednesday, March 31) in Ballroom III at Where 2.0.