There are heaps of well-written solutions for interacting with Twitter’s REST API, but most are overly technical for non-developers. We wanted to build something that a disaster response technical team could roll out during a hurricane or major earthquake. So we created a lightweight Node.js program to mine Twitter and serve data that you can host anywhere. Today, we’re tagging our new twitter-server application with v0.1.

Twitter-server v0.1 was designed for simplicity, it has a solid Express.js foundation, serves static files, and is now ready for development.

Serving Dynamic Content Through Static Files

We love static files. They’re cheap to host and easy to maintain. With the proper safeguards, it becomes very difficult to take down your service. Twitter-server is built upon Node.js to serve dynamic content through static files. We decided to go this route because we didn’t want to worry about scaling up and down during a disaster.

To serve content, Twitter-server queries Twitter every 10 seconds or so for a list of updates. This keeps the application within the bounds of Twitter’s API limit. We also avoid the need for the user to sign into Twitter herself. The application can query a list of Twitter users, or do a keyword search.

The search parameters are currently hard-coded in a config file. This provides a foundation for making improvements. Our next step is to attach them to a Google Spreadsheet so they can be changed on the fly.

Once the Node.js application gets its response from Twitter, it spits out a simplified JSON file that is immediately pushed to an Amazon S3 bucket. Since it updates every 10 seconds, it continually over-writes the last file, so when new users visit your page, they get fresh Tweets.

For our purposes, the app lives on a free Amazon EC2 instance, where it happily chugs away. You could host it anywhere though, and of course you don’t need to use S3 either.

All this work has been in collaboration with our friends at ReliefWeb and their upcoming improved API. Look for updates in the coming week and how this Twitter application can be used to track real-time disasters. Twitter-server is also open source, and the code is available on Github for you to use. A good chunk of the codebase is forked from @Kamicut’s Seattle repo, which connects to Twitter’s Streaming API and serves data through Websockets. Join the development!