Integrating SlingshotSMS with Managing News
Lightweight systems for data collection meet powerful tools for analytics
We can now dynamically map and visualize real time SMS messages in Managing News using the new SlingshotSMS feature. All the code is available on github. This release is particularly exciting for us because it ties together two of our core projects in such a way that each is made better. SlingshotSMS is a lightweight SMS gateway that can be run off of a USB drive, needing only a GSM modem and an internet connection to act as a bridge between mobile phones and the web. Managing News is a powerful data aggregator and visualization tool that lets distributed teams work together to make large amounts of information useful. Together, they provide an extensible framework for teams conducting mobile data collection projects in the field.
Extensibility is key here because we need this to meet a wide variety of use cases in order for it to be useful. We have been particularly focused on use cases related to election monitoring, but this is just one of many possible applications. Here's a graphic that Saman made illustrating how the system works:
This technology is meant to accompany your existing processes of data collection. You have people in the field, they have cellphones, you have a phone back in headquarters, and they can text in messages to you that are then relayed to a visualization space, which helps keep you and your team on the same page.
Since SlingshotSMS runs on a USB drive, you just plug it in, plug in your phone, and set up what website you want to have the SMS messages sent to. The SMS messages are turned into RSS 2.0 and PUSHed, like as a fat ping. You computer just needs internet to send these messages.
Going back to this election monitoring example, here you see the election monitor is texting in that the polling station is closed. You'll notice that the text message contains a few things: a polling station ID, the word "closed", and the word "security." These are key terms we are going to want to look for on the Managing News side to flag.
SlingshotSMS just pushes the data up to Managing News. The Managing News site will have a custom parser that will break up this text message, pulling out key words and numbers.
Customizing SMS parsing for each project
Out of the box, the Managing News SlingshotSMS bridge simply accepts the SMS and incorporates it into the default Managing News workflow, ignoring important information in this example like the fact the polling station is closed and there is a security issue. This is where the pluggable nature of Managing News proves its worth. It is simple to write a custom parser that replaces the default parser that ships with Managing News. With some creative use of the Drupal taxonomy system to filter incoming results into Managing News channels and some very basic regex, you can quickly have a system that is able to capture this data and let Managing News users react to it. We'll publish a blog post soon explaining exactly how to do this.
Authentication
Authentication of incoming SMS messages is vital in these situations, and we took extra care to make sure the Managing News Slingshot feature will only receive data from authorized sources. The framework is dependent on the Drupal KeyAuth module which allows signed messages to pass between a SlingshotSMS installation and Managing News. When setting up the Managing News Slingshot feature, users are given public and private keys that they then copy into the SlingshotSMS configuration file. In the future we are considering switching to an OAuth based solution.
To get set up, you'll need the following:
- The latest version of Managing News, Beta 7
- The latest release of SlingshotSMS - mac_keyauth_04.zip (Mac only for now)
- The Keyauth Module
- The Managing News Slingshot integration feature
Other related blog posts:
- Managing News Beta 7 Released
- SlingshotSMS 0.2 Released: Now Providing RSS 2.0 and Modem Autodetection
- Parsing and Storing Data from SMS Messages
- SlingshotSMS Alpha Code Released: A Lightweight SMS Gateway on a Stick
- Dry Run of SlingshotSMS for Situation Monitoring
- Mobile Data Collection Workshop Gets Decision Makers Collaborating with Technologists
Thank you so much. This is an
Thank you so much. This is an excellent product.
Very much looking forward to reading about the custom parser with regex.
Having some way of automatically scoring and assigning feed items into channels linked to Drupal’s taxonomy will take this product to a whole new level.
Awesome work!
Awesome work!
Porting to Linux or Windows
Any chance SlingshotSMS is gonna b ported to Linux and/or Windows soon? Mac use especially as a server isn’t very frequent in the Middle East :(
Definitely
Absolutely! SlingshotSMS is developed for Mac, Windows, and Linux, and can be run from source in any of the three – this release was packaged first for Mac. On a linux machine, just get the source from github, install the dependencies, and it’ll run just like on mac. Same with Windows, although I’ll have a .exe-packaged release online very soon, which packages the dependencies just like the Mac release.
automatic tagging
Your next post will deal with filter and dispatch incoming results into Managing News channels with regular expression, that’s great.
I would also like to see what features have been build to use opencalais and other semantic services.
I would also look forward to have some sort of community website as for openatrium. Something were we could see features build on top of managing news.
Managing News intergration with Yahoo Placemaker
Robert Soden is currently working on integration with Yahoo! Placemaker for auto geo tagging, this is key for when we are working on a non-English site, in this case we are doing a Managing News in Chinese for Teachers Without Boarders, and for the geo tagging to work place maker helps a lot. I am sure Robert will be posting on that as this develops and if we like how this comes together we will of course release this on github.
Awesome
Great work! Can’t wait to see it out in the field and collecting data.
Thank you! Looking forward
Thank you!
Looking forward to your next article about custom parser.
Also, I’ve reported a bug to SlingshotSMS GitHub preventing cyrillic sms messages from being sent or received.
Please know, your work is much appreciated all over the world.
Hey Alexander, Absolutely –
Hey Alexander,
Absolutely – supporting utf8 text properly is next on the list for SlingshotSMS bug fixes. Like most other things, a lot of the work has been done upstream by pygsm, so it’s a matter of integrating, testing, and packaging.
Awesome work! Scary pictures.
Awesome work! Scary pictures. I love it: Men with machine guns = “security issue”.