This past weekend Jue and I presented at the Online News Association Conference in Chicago. We shared what we’ve learned from creating mapping sites that toggle through a high volume of data: specifically, how we mapped tens of thousands of rows of data on a series of maps using a mostly-front-end stack.

We built this data browser with Backbone and Leaflet. Given the amount of data, we incorportated data binding and drawing methods from D3.js on top of Leaflet to boost the loading and interaction speed of the site. By writing python code to output configuration and data json files, we made the site modular. We customized all the geography files into efficient topojson — some with necessary simplications to control file size, and others with customized data field to be joined with the rest of datasets in the front-end. From this experience we learned:

• The good: we made a site that is hosted on Github. Even though the site handles tons of data, it will rarely go down. Topojson makes geography in the browser faster than ever.

• The crazy: we spent a lot of brain cells preparing the data and its metadata in python, in order to unload the javascript from heavy calculations.

Like other problems we encounter at Development Seed, we try to tailor our approach using the most fitting technology. We are constantly assessing our tech stacks, which prepares us for challenges that come in all shapes and sizes. Although we’ve managed the complexity of this project well, we are also excited about geographic databases such as PostGIS, which we may plug in the next time we come across a project of this size.

I also did a hands-on session about designing maps using d3 and topojson. We provided several examples to help those starting to learn this workflow. You can find the talk here. We’ve found this workflow to be valuable, since it allows us to serve complex sites as static files. This is consistent with our approach to building interactive CMS-Free sites.

We hope you find our presentations useful. Let us know if you have questions or comments.