Node.js - Development Seed http://developmentseed.org/blog/nodejs The latest developments in evented, server-side javascript. en NodeDC Meetup Tonight at 7 pm http://developmentseed.org/blog/2012/may/16/nodedc-may <p>The monthly <a href="http://nodedc.github.com/">NodeDC meetup</a> is <strong>tonight (Wednesday, May 16) at 7:00 pm</strong> at <a href="http://stetsons-dc.com/">Stetson's Bar and Grill</a>, near the intersection of 16th and U streets NW here in Washington, DC. This is a great opportunity to meet local node.js developers and hear what they're working on, and in general to learn more about the popular <a href="http://nodejs.org/">platform</a> for building fast, scalable applications.</p> <p>There will be a round of introductions starting at 7:30 pm, followed by two lightning talks:</p> <ul> <li><p><a href="http://twitter.com/johnkelvie">John Kelvie</a> and Michael Myers of Atomizer Software will present the technical design of a game they are building with node.js, and request feedback.</p></li> <li><p><a href="https://twitter.com/#!/studgeek">David Rees</a> will talk about <a href="http://derbyjs.com/">Derby.js</a>, a MVC framework for writing realtime, collaborative applications that run in Node.js and browsers, and compare it to <a href="https://github.com/meteor/meteor">Meteor</a>.</p></li> </ul> <p>After the talks, plan to stick around for more conversations over drinks.</p> <p>NodeDC meets monthly, usually on the third Wednesday of the month. For updates, watch <a href="http://nodedc.github.com/">node.github.com</a> on github, follow <a href="https://twitter.com/#!/nodedc">@nodedc</a> on Twitter, and join the <a href="http://www.meetup.com/node-dc/">meetup group</a>. See you tonight!</p> <p><img src="http://farm6.staticflickr.com/5192/7209879542_d1781b82ae.jpg" alt="" /></p> 2012-05-16T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2012/may/16/nodedc-may NodeDC and DCjQuery Joint Meetup Tonight http://developmentseed.org/blog/2012/feb/20/nodedc-meetup-tonight <p>Tonight, <strong>Monday, February 20</strong> at 7:00 pm, we're holding a <a href="http://nodedc.github.com/">NodeDC meetup</a> at <strong>Fathom Creative</strong> at 1333 14th Street NW (at Rhode Island). This month we're teaming up with the <a href="http://www.meetup.com/DC-jQuery-Users-Group/">DCjQuery folks</a> to bring together a bigger crowd who is interested in all things JavaScript. There are three lightning talks on the lineup:</p> <ul> <li><p><a href="https://twitter.com/#!/dylang">Dylan Greene</a> will present <a href="http://en.wikipedia.org/wiki/Canvas_element">HTML Canvas</a> and show how he's using it on the drawing game <a href="http://doodleordie.com/">DoodleOrDie</a>.</p></li> <li><p><a href="https://twitter.com/#!/ryan_roemer">Ryan Roemer</a> will talk about <a href="https://github.com/caolan/async">async.js</a>.</p></li> <li><p><a href="https://twitter.com/#!/jafaramjad">Jafar Amjad</a> will present on making a Node.js multiplayer site, specifically one that uses a low amount of bandwidth per player and features lasers, gravity, and walk cycles via animated gif’s. Check out his <a href="http://jaf.ar.com/">early beta version</a> for a preview.</p></li> </ul> <p>It should be a fun time to meet Node.js, jQuery, and JavaScript developers, share experiences, and learn what everyone is up to. Grab a six pack or some snacks to contribute to the group, and come on out tonight at 7:00 pm.</p> <p>NodeDC holds a meetup once a month, usually on the third Monday of the month. Watch the <a href="http://nodedc.github.com/">website</a> and follow <a href="http://twitter.com/nodedc">@NodeDC</a> on Twitter for updates.</p> 2012-02-20T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2012/feb/20/nodedc-meetup-tonight NodeDC Meetup Tonight at 7:00 pm http://developmentseed.org/blog/2011/12/14/nodedc-meetup-december <p>Today - Wednesday, December 14 - starting at 7:00 pm at <a href="http://stetsons-dc.com/">Stetson's</a> is the <a href="http://nodedc.github.com/">NodeDC meetup</a>. These monthly meetups bring together developers to talk about <a href="http://nodejs.org/">node.js</a> - the platform for building fast, scalable applications that we use for our <a href="http://developmentseed.org/projects/">data and mapping sites</a>. It's a great opportunity to share what you are working with in node.js, find out how others are using it, and get to know other developers in the city. Tonight there will be three lightning talks:</p> <ul> <li><p><a href="http://developmentseed.org/team/ian-ward">Ian</a> will talk about how we deploy node apps here at Development Seed, flying through components like SSL, Upstart, Nginx, logging, and monitoring.</p></li> <li><p><a href="http://developmentseed.org/team/will-white">Will</a> will talk about node-inspector, a browser-based node debugging tool that allows you to set breakpoints, step though code, and edit code while a process is running.</p></li> <li><p><a href="https://twitter.com/#!/raydaly">Ray Daly</a> will talk about concept of TTD for APIs using node and hopes to get feedback from the group.</p></li> </ul> <p>The NodeDC meetups are held once a month on the third Wednesday of the month, with next month's on <strong>Wednesday, January 18</strong>, so mark your calendars. These meetups are open to everyone, with people of all levels of node.js and programming expertise welcome.</p> <p>Check out <a href="http://nodedc.github.com/">nodedc.github.com</a> for more details, and follow <a href="http://twitter.com/nodedc">@NodeDC</a> on Twitter for updates.</p> 2011-12-14T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2011/12/14/nodedc-meetup-december Node.js Meetup This Monday in Washington, DC http://developmentseed.org/blog/2011/nov/11/nodejs-meetup-monday-washington-dc <p>This Monday, November 14 is the <a href="http://nodedc.github.com/">November Node.js meetup</a> here in Washington, DC. These meetups bring together <a href="http://nodejs.org/">node.js</a> developers and people interested in learning more about the server-side JavaScript environment, best known for its speed and flexibility, to talk, show off their work, and ask questions. As before, the meetup will feature a series of quick, five minute lightning talks on anything relating to node.js. This month, four people have volunteered to present:</p> <ul> <li><p><a href="https://twitter.com/#!/younghahn">Young Hahn</a> will talk about <a href="http://couchdb.apache.org/">CouchDB</a>, an open source document-oriented database written mostly in Erlang, highlighting tricks and other useful things he's learned about best utilizing it in conjunction with node.js.</p></li> <li><p><a href="https://twitter.com/#!/cvee">Chris Verwymeren</a> will talk about running node.js apps in Ubuntu with <a href="http://upstart.ubuntu.com/">Upstart</a>.</p></li> <li><p><a href="http://twitter.com/#!/Zugwalt">Aaron Silverman</a> and <a href="https://twitter.com/#!/dylang">Dylan Greene</a> will demo their <a href="http://blog.nodeknockout.com/post/9895206272/node-knockout-winners">Node Knockout</a> project <a href="http://doodleordie.com/">DoodleOrDie</a>, a drawing guessing game a la pictionary, explaining how they built it in just eight hours.</p></li> <li><p><a href="https://twitter.com/#!/incanus77">Justin Miller</a>, in town from Portland, will talk about how he bridged Cocoa to node.js in the map design tool <a href="http://mapbox.com/tilemill">TileMill</a>.</p></li> </ul> <p>The meetup will be held in the upstairs bar at <a href="http://stetsons-dc.com/">Stetson's</a> and will get started at 7:00 pm, with presentations starting at 7:30 pm after a round of introductions. The NodeDC group meets once a month, usually on the third Wednesday of the month. You can see what was discussed at past meetups at <a href="http://nodedc.github.com/">nodedc.github.com</a>, and for updates follow <a href="http://twitter.com/#!/nodedc">@nodedc</a>.</p> 2011-11-11T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2011/nov/11/nodejs-meetup-monday-washington-dc Discussing Underscore.js at Tuesday's Node.js DC Meetup http://developmentseed.org/blog/2011/09/14/discussing-underscorejs-nodejs-dc-meetup <p>At next week's <a href="http://nodedc-september.eventbrite.com/">Node.js meetup here in Washington, DC</a>, I'll give a brief introduction to <a href="http://documentcloud.github.com/underscore">Underscore.js</a>, a functional programming library for Javascript. At its base, Underscore.js is about data structures (usually arrays) and interacting with them more efficiently. It also forms the basis or inspiration for several other popular libraries, such as <a href="http://documentcloud.github.com/backbone">Backbone</a>, and learning to make use of it properly will allow you to segue this knowledge into other interesting areas.</p> <p>While this library has traditionally been associated with the browser environment, it is still widely used and very applicable to the server side of the equation. As with most node modules, Underscore.js can be installed using the <a href="http://npmjs.org">npm package management system</a>. It's the <a href="http://search.npmjs.org/">single most depended on package available on npm</a>, something that illustrates just how many people consider it a basic part of their Javascript programming arsenal.</p> <p>I'll go into more detail at the <a href="http://nodedc-september.eventbrite.com/">Node.js meetup</a> happening on <strong>Tuesday, September 20 at 7:00 pm</strong> in our garage here in Washington, DC. I'm also including more details on Underscore below for those who can't wait.</p> <h2>Some underscore basics</h2> <p>A common situation in which you might use Underscore.js is for retrieving records from your datastore of choice (we prefer CouchDB, but it could be whatever).</p> <pre><code>var countries = [ { "iso3": "AFG", "name": "Afghanistan"}, { "iso3": "ALB", "name": "Albania", "active": true }, /* trimmed for brevity */ { "iso3": "ZMB", "name": "Zambia" }, { "iso3": "ZWE", "name": "Zimbabwe", "active": true } ] </code></pre> <p>Say you want to filter the list of countries to only include the items that have the active property set to true. This can be easily done through the following code that makes use of the <a href="http://documentcloud.github.com/underscore/#select">select method</a>, which will step through your objects and pass each one to the test function you supply, returning a new array containing only the objects that passed the test.</p> <pre><code>var activeCountries = _(countries).select(function(country) { return country.active === true; }); </code></pre> <p>Now say you want to have just the id or iso3 code, so you can make use of the array to do input validation. Underscore provides the wonderfully useful <a href="http://documentcloud.github.com/underscore/#pluck">pluck method</a> which will step through your objects and return an array containing the values of the property you requested.</p> <pre><code> // Build an array of only valid country codes var validISO3 = _(activeCountries).pluck('iso3'); </code></pre> <p>Then you can use the <a href="http://documentcloud.github.com/underscore/#include">include method</a> to test for the validity.</p> <pre><code>// check if a submitted value is allowed : var success = _(validISO3).include(req.params.iso3); </code></pre> <p>These are just some of the basic things you can do with the library. It is filled with little gems that help you do all kinds of programming tasks. The entire library fits into a 3k file when minimized, and considering its small size and inherent power, there's really very little reason not to make use of it both on the server and the client side.</p> 2011-09-14T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2011/09/14/discussing-underscorejs-nodejs-dc-meetup Join Us at the DC Node.js Meetup on July 18th http://developmentseed.org/blog/2011/jul/06/join-us-dc-nodejs-meetup-july-18th <p>The first <a href="http://nodedc.eventbrite.com/">DC Node.js Meetup</a> is coming up on Monday, July 18, and we're happy to be hosting it in our <a href="http://www.flickr.com/photos/diggersf/5852812543/in/photostream">newly enhanced</a> <a href="http://www.flickr.com/photos/developmentseed/5884120113/in/photostream">garage</a>. We are using <a href="http://nodejs.org/">Node.js</a> in <a href="http://tilemill.com/">all</a> <a href="http://mapbox.com/#/">of our</a> <a href="http://developmentseed.org/projects">projects</a>, and we're hoping that this meetup will bring together other developers using Node.js so we can all share stories, learn from each other, and have fun over some beers.</p> <p><a href="http://nodedc.eventbrite.com/"><img src="http://farm6.static.flickr.com/5312/5885655060_70deaa854e.jpg"></a></p> <p>We'll have a series of five minute lightning talks about topics related to Node.js. Here's the lineup so far:</p> <ul> <li><strong>An Intro to Node.js</strong>: A quick rundown of the basic principles of Node.js and what types problems it's meant to solve. We'll record this one so we won't have to repeat it at future events.</li> <li><strong>Setting up a node development environment with nvm and npm</strong>: Developing applications with Node.js begins with setting up a development environment. Learn how <a href="https://github.com/creationix/nvm">nvm</a> can help you manage multiple versions of node and how <a href="http://npmjs.org/">npm</a> can help you use libraries from other developers and share your own.</li> <li><strong>Demo of the <a href="https://github.com/rektide/requisitioner">requisitioner</a> module</strong>: Requisitioner takes node style modules and wraps them to make them compatible with <a href="http://wiki.commonjs.org/wiki/Modules/AsynchronousDefinition">Asynchronous Module Definition</a>, and usable in a web browser.</li> </ul> <p>We have room for a few more talks. If you'd like to present, send your topic to <a href="http://twitter.com/developmentseed">@developmentseed</a> or info@developmentseed.org. We'll have a projector, screen, and internet, and slides and demos are welcome.</p> <p>The meetup will kick off at 7:00 pm, and we'll start the lightning talks at 7:30 pm after a round of introductions. Bring some beer/drinks to share with the group - we'll provide the fridge and some snacks. <a href="http://nodedc.eventbrite.com/">RSVP here</a> and hope to see you there!</p> <div id='ts-embed-1309963887654-script'><script src='http://tiles.mapbox.com/aj/api/v1/embed.js?api=mm&amp;size%5B%5D=530&amp;size%5B%5D=300&amp;center%5B%5D=-77.03160524368218&amp;center%5B%5D=38.91313942135115&amp;center%5B%5D=16&amp;layers%5B%5D=devseed-hq-crop&amp;options%5B%5D=zoompan&amp;options%5B%5D=zoomwheel&amp;el=ts-embed-1309963887654'></script></div> 2011-07-06T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2011/jul/06/join-us-dc-nodejs-meetup-july-18th Opening Data on 7,500 Higher Education Institutions and Almost 14,000 U.S. School Districts http://developmentseed.org/blog/2011/jun/16/open-data-site-compares-schools-65-different-indicators <p>The Federal Education Budget Project at New America Foundation just relaunched its open data site <a href="http://febp.newamerica.net/">EdBudgetProject.org</a>, adding new mapping and comparison tools for more than 65 indicators about the 14,000 public school districts in the United States and an entire new section monitoring 7,500 higher education institutions.</p> <p>The site features fast auto-complete search that allows you to quickly find any school district or higher eduction institution right from the homepage. Each of the two datasets includes state rankings, showing in a glance how states compare. The maps on the site have also been enhanced with chlorophleth displays, improved interactions, and more appealing base maps.</p> <p><img src="http://farm4.static.flickr.com/3081/5839227705_22b9044c6c.jpg" alt="Compare public school districts" /></p> <p>The new higher education dataset includes information about federal grants and loans (like as Pell Grants and Work-Study), graduation rates, and tuition and fees. This information is available for both individual institutions and in aggregate at the state level.</p> <p><img src="http://farm3.static.flickr.com/2747/5839778526_8799c9deb7.jpg" alt="Compare colleges and universities" /></p> <p>The <a href="http://developmentseed.org/blog/2009/may/13/opening-education-data-funding-demographic-and-achievement-data">new iteration</a> of the site also features updated data browsing and mapping tools. The site is intended to allow natural exploration the data. As you view information about a school or state, nearly any indicator can be used as the basis of a comparison which visualizes how that school or state against similar ones. You can also dig down into fairly complex comparisons using the site's graphing tools.</p> <p><img src="http://farm6.static.flickr.com/5108/5839227783_bc1dac6b39.jpg" alt="Flexible data comparison tools" /></p> <p>We also completely overhauled the administrative backend of the site. All of the data in the site is managed externally, with the bulk of it imported in batches. An issue in the previous implementation was that it was possible (though not likely) that the site's database would import data incompletely if a page was requested while data was being added or if the import failed. Additionally, it wasn't possible to review the imported data before it was available to the general public. So any formatting or precision issues would be immediately visible, and require a re-import or database rollback to fix.</p> <p><img src="http://farm4.static.flickr.com/3075/5839778050_3a1646ce36.jpg" alt="A view of the backend" /></p> <p>With the new version of the site, we implemented a versioned dataset manager. Each of the four datasets in the site (public schools state level, public school districts, higher education state, higher education institutions) can have any number of versions of data in the site at a time. There is always a version of each set to active, and it's possible to set one to 'preview' so that it's visible to administrators only.</p> <p>Metadata about each dataset is managed in a similar way. For every indicator in a dataset there is a set of details that the site needs to know, from things like a title and description to the required formatting. This information is captured in a schema document. In exactly the same manner as the data itself, these schema documents can have any number of versions available in the site, and a particular one is set as active for each of the four data sets. Previewing also works for schemas, allowing administrators to verify that meta data and formatting changes are proper before a schema is live. This flexibility in managing and previewing data sets and related schema information makes administering the site a more reliable experience.</p> <p>The site itself was build using <a href="http://expressjs.com/">Express</a> and <a href="http://nodejs.org">Node.js</a>. The administrative backend leverages <a href="http://socket.io">socket.io</a> to provide real-time updates about data imports and validation. The data is stored in <a href="http://couchdb.apache.org">Couchdb</a>, and the site uses <a href="http://elasticsearch.org">Elastic Search</a> to index all the data.</p> 2011-06-16T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2011/jun/16/open-data-site-compares-schools-65-different-indicators Regular Node.js meetups in DC? http://developmentseed.org/blog/2011/may/06/regular-nodejs-meetups-dc <p>It was awesome to see <a href="http://www.flickr.com/photos/developmentseed/5687397753/in/photostream">almost 50 people</a> pack into the <a href="http://www.meetup.com/DC-jQuery-Users-Group/events/16289276/">DC jQuery meetup</a> at <a href="http://fathomgallery.org/">Fathom Gallery</a> last week with six packs in hand to talk about <a href="http://nodejs.org/">Node.js</a>. The conversation ranged from the classic make your <a href="http://nodejs.org/">hello world Node.js</a> server by Jonathan Altman, to <a href="http://developmentseed.org/team/jeff-miccolis">Jeff</a> sharing some Node.js sites we have in production like <a href="http://pakistansurvey.org/">PakistanSurvey.org</a> and <a href="http://mapbox.com/#/">MapBox.com</a>, to <a href="http://developmentseed.org/team/will-white">Will</a> demoing tools we've built on Node.js like <a href="http://tilemill.com/">TileMill</a> and <a href="https://github.com/mapbox/tilestream">TileStream</a>. A big hat tip to <a href="https://twitter.com/#!/GregLavallee">Greg Lavallee</a>, the organizer of the <a href="http://www.meetup.com/DC-jQuery-Users-Group">DC jQuery meetup</a>, for organizing a great event that really showed how much energy there is in Washington, DC around Node.js.</p> <p>This raises the question, should we start having a dedicated Node.js monthly meetup? Would folks and shops in town be interested in this? We are excited about the idea with all our development now on Node.js and, after <a href="http://developmentseed.org/blog/2011/may/04/nodeconf-tomorrow-just-arrived-portland">more than half our team attended the first ever NodeConf</a> yesterday in Portland, we are fired up. We would love to play a leadership role in organizing these <em>(read finding a bar to meet in)</em> if folks are interested in starting them.</p> <p>If you're in Washington, DC and would be interested in Node.js meetups, please let us know! I'm looking forward to hearing people's feedback and ideas for starting these.</p> 2011-05-06T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2011/may/06/regular-nodejs-meetups-dc NodeConf Tomorrow: Just Arrived in Portland http://developmentseed.org/blog/2011/may/04/nodeconf-tomorrow-just-arrived-portland <p>More than half of <a href="http://developmentseed.org/team">our team</a> just landed in Portland for <a href="http://nodeconf.com/">NodeConf</a>, a one day conference all about <a href="http://nodejs.org/">Node.js</a> that is happening tomorrow. With all of our work now built in Node.js, including <a href="http://mapbox.com/">our mapping products</a> and our <a href="http://developmentseed.org/projects">data visualization projects</a>, we are excited to attend and meet other lead developers in person and talk about where the platform is headed.</p> <p>You know what else we're excited about? Bunk beds :) We'll be taking over a private room at the <a href="http://www.yelp.com/biz/northwest-portland-international-hostel-portland">Northwest Portland International Hostel</a> that has four sets of bunk beds and doing it up summer camp style. The hostel is just a few blocks from the conference venue as well as some restaurants and bars, and most of the team will be there, with the exception of Justin who lives in Portland and Alex who is staying nearby. If you're at NodeConf and want to meet up, look for us or get in touch via Twitter: <a href="https://twitter.com/#!/tmcw">Tom</a>, <a href="https://twitter.com/#!/younghahn">Young</a>, <a href="https://twitter.com/#!/willwhitedc">Will</a>, <a href="https://twitter.com/#!/kkaefer">Konstantin</a>, <a href="https://twitter.com/#!/lxbarth">Alex</a>, <a href="https://twitter.com/#!/miccolis">Jeff</a>, <a href="https://twitter.com/#!/springmeyer">Dane</a>, <a href="https://twitter.com/#!/incanus77">Justin</a>, <a href="https://twitter.com/#!/dhcole">Dave</a>, and <a href="https://twitter.com/#!/ericg">Eric</a>.</p> <p>For those new to Node.js, the platform is known for its speed and for being lightweight, and it has a great decentralized community connected through github. Over the past six months, we have been releasing our Node.js code on <a href="https://github.com/developmentseed">github.com/developmentseed</a> and <a href="https://github.com/mapbox/">github.com/mapbox</a>. If speed matters for the apps that you build, we highly recommend looking into Node.js.</p> <p>The <a href="http://nodeconf.com/schedule.html">full schedule for NodeConf is here</a>, and we'll post back with a link to video if sessions are recorded.</p> 2011-05-04T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2011/may/04/nodeconf-tomorrow-just-arrived-portland Data Browser Shows Final 2010 Wolesi Jirga Election Results in Afghanistan http://developmentseed.org/blog/2010/dec/23/data-browser-shows-final-2010-wolesi-jirga-election-results-afghanistan <p>The final results from Septembers's Wolesi Jirga elections in Afghanistan have been released on <a href="http://2010.afghanistanelectiondata.org/">2010.afghanistanelectiondata.org</a>. The data browser provides a user interface that lets you explore election results from a country-wide level and then quickly drill down by province and district, all the way to the polling center and candidate level. While the final winners were announced by the <a href="http://www.iec.org.af/eng/">Independent Election Commission (IEC)</a> on November 24th, today is the first time the raw data is available to the public, thanks to the open data team at the <a href="http://www.ndi.org/">National Democratic Institute (NDI)</a>.</p> <p><img src="http://farm6.static.flickr.com/5086/5285890663_f49d2b63c9_z.jpg" alt="Hirat | 2010 Wolesi Jirga Election" /></p> <p>This final set of data includes 3.9 million valid votes from the 4,691 polling centers that participated in the election, with results coming from all 34 provinces in Afghanistan and 370 out of 400 districts. The dataset does not include the 1.3 million votes that the IEC had invalidated prior to the release of the preliminary data on October 20th (almost a quarter of the nearly 5.6 million ballots cast) or the 238,542 votes that they invalidated between the October preliminary results and the final results from the end of November.</p> <p><img src="http://farm6.static.flickr.com/5169/5285890615_5717b11fb5.jpg" alt="Percentage of Stations Reporting | 2010 Wolesi Jirga Elections" /></p> <p>The site also provides comparisons between the final data and the interim data. For example between October and November, 238,542 ballots (6%) were discarded country-wide with some provinces significantly affected - like in Zabul and Nuristan where 67% and 40% respectively of the preliminary valid ballots were thrown out. The largest number of discarded ballots was 63,564 in Hirat, accounting for over 18% of preliminary votes. Other interesting comparisons between October 20th's preliminary results and November 24th's final results include:</p> <ul> <li>30 leading candidates were disqualified, or 12% of the 249 delegates.</li> <li>10 candidates were disqualified in Hirat, causing six of the 17 seats to change hands between the preliminary and final results.</li> <li>Bamyan, Nimroz, and Panjsher provinces had no changes at all</li> </ul> <p>The raw data from both preliminary and final results is available to download in csv format from <a href="http://afghanistanelectiondata.org/open/data">afghanistanelectiondata.org/open/data</a> and there are a number of KML, tilesets, and pdf maps showing the data at <a href="http://afghanistanelectiondata.org/open/maps">afghanistanelectiondata.org/open/maps</a>.</p> <h2>Building for speed</h2> <p>The site is built with <a href="http://nodejs.org/">Node.js</a> and <a href="http://expressjs.com/">Express</a>, and the data is stored in <a href="http://couchdb.apache.org/">CouchDB</a>. Node.js is ideal for a data-heavy site like this because it allows us to show an enormous amount of dynamic and complex data on each page without sacrificing speed or relying on greedy caches that can go stale. CouchDB worked well for this project because we needed to use one dataset, but display it in many different ways. CouchDB's system of design documents and query by MapReduce allows it to transform data very efficiently.</p> 2010-12-23T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2010/dec/23/data-browser-shows-final-2010-wolesi-jirga-election-results-afghanistan Data Browser Shows Views In Pakistan's Tribal Regions http://developmentseed.org/blog/2010/oct/20/data-browser-shows-views-inside-pakistans-tribal-regions <p>This morning the Counterterrorism Strategy Initiative at the New America Foundation released <a href="http://www.pakistansurvey.org/">PakistanSurvey.org</a>, opening data from 1,000 face-to-face interviews across 120 villages in Pakistan's northwest Federally Administered Tribal Areas (FATA). The site is designed to let users quickly drill down and thin slice survey data and read agency-specific analysis from regional experts. This is the first comprehensive public opinion survey done in the region, and it is mashed with a mapping of 142 reported drone strikes in FATA through July 2010 to add additional context.</p> <p><img src="http://farm2.static.flickr.com/1331/5099492978_57d9b4c192.jpg" alt="Inside Pakistan's Tribal Regions, a look at the front landing page" /></p> <p>The architecture focuses on showing disaggregates for each of the seven agencies in FATA and breakouts for each survey question, allowing you to compare specific opinions across different agencies. Every response on both agency and question pages can be filtered by demographic data, gender, age, education, marital status, and income level.</p> <h2>Administrative Agencies</h2> <p>Each agency page exposes a full listing of local survey responses and a zoomed in map view of the area showing drone strikes. <a href="http://developmentseed.org/team/aj-ashton">AJ</a> designed the the drone strike overlay map by who authorized the strikes, Bush (in orange) or Obama (in yellow), and scaled the strike points based on the estimated number of militants and non-militants killed.</p> <p>Below is a screenshot from the <a href="http://pakistansurvey.org/agency/north-waziristan">North Waziristan overview page</a>, which is the second-largest agency in FATA and, as you can see from the drone strike data on the map, the center of U.S. drone operations in Pakistan. On the right hand side, there is a filter that adjusts the graphs by demographics.</p> <p><img src="http://farm2.static.flickr.com/1380/5099493016_4ea6565ed8_z.jpg" alt="Screenshot showing mapping drone strikes in North Waziristan" /></p> <h2>Sentiment Survey Questions</h2> <p>All of the 43 substantive questions from the survey are broken out to show both FATA wide-sentiment and sentiment at the agency level. The graphs are designed to make comparisons while skimming a large amount of data really fast. Here is a look at the multi-part question "<a href="http://pakistansurvey.org/question/suicide-bombings">Are suicide bombings justified against the following targets?</a>" Like on the agency pages, you can filter responses by demographic.</p> <p><img src="http://farm5.static.flickr.com/4091/5098893285_03871ae381.jpg" alt="Screenshot showing responses to &quot;Are suicide bombings justified against the following targets?&quot;" /></p> <h2>Designing for Speed and Using Open Source</h2> <p>We built this site on top of <a href="http://expressjs.com/">Express</a>, a very fast and small server-side JavaScript framework built on <a href="http://nodejs.org/">Node.js</a> and <a href="http://github.com/senchalabs/connect">Connect</a>. For those new to Node.js, it is a low level toolkit designed for writing high performance server-side JavaScript applications. We used <a href="http://www.mongodb.org/">MongoDB</a> for the database. The maps were made using <a href="http://openlayers.org/">OpenLayers</a>, and they are all hosted on <a href="http://mapbox.com/">MapBox.com</a>, leveraging some of the newer tools we're building like <a href="http://github.com/developmentseed/TileLive">TileLive</a>, which <a href="http://developmentseed.org/blog/2010/oct/13/adding-interactivity-complex-data-heavy-maps-drupal">Tom blogged about last week</a>.</p> <p>For this site, we needed to show a lot of data together without sacrificing a fast experience. Speed really matters when you are building interactive data sites like PakistanSurvey.org, and our goal here was to encourage a lot of browsing and filtering.</p> <h2>Background Information on the Survey</h2> <p>The original survey was conducted from June 30 to July 20, 2010 and has a margin of error of +/- 3 percent. The full methodology is available on <a href="http://www.pakistansurvey.org/about/methodology">pakistansurvey.org/about/methodology</a>. There you can also find details about how the <a href="http://pakistansurvey.org/about/drones">drone strike data</a> was gathered.</p> <p><img src="http://farm5.static.flickr.com/4149/5099493068_ac4c4982cb.jpg" alt="A look at the increase in drone strikes" /></p> <p>The site was launched this morning at the the <a href="http://www.usip.org/">United States Institute of Peace</a> by <a href="http://www.newamerica.net/people/peter_bergen">Peter Bergen</a>, the co-director of the Counterterrorism Strategy Initiative at<a href="http://newamerica.net/"> New America Foundation</a>, the team that lead the survey work.</p> 2010-10-20T00:00:00+00:00 Development Seed http://developmentseed.org/blog/2010/oct/20/data-browser-shows-views-inside-pakistans-tribal-regions