Comment Aggregation: Tightly Integrating Third Party Social Networks with Your Site


3 min read

For the Knight Pulse website we recently built, we came up with a simple way of embedding video and comments from Vimeo in a blog post. Here’s how simple it is. Just copy and paste the URL of the video to the node form, submit it, and you’re done.

Watch this screencast to take a look at the UI:

This feature is of great value because it creates a bridge between each video’s two presences — the conversations happening on Knight Pulse and the conversations on Vimeo. In this post, I’ll show you how we built the feature and how you can do it yourself for Vimeo and also other social networks like Flickr, YouTube, and, with a little bit of homework, almost any other site.

Embedding the Video

The first part is simple. Embedded Media Field (awesomely) supports a number of media sites and conveniently enough one of them is Vimeo. Install and configure CCK and Embedded Media Field for a content type and then we’re ready to embed video just by copying/pasting the URL from the video’s page.

Act 2: Pulling in Comments

This is the part we wrote from ground up. FeedAPI Comments aggregates comments to either Drupal comments or nodes. In our case, we would like to pull in comments right on the node that’s embedding the video. That means we enable FeedAPI on this node’s content type and enable the Comments parser (part of FeedAPI Comments) and the FeedAPI Comments itself.

Act 3: The Glue

At this point, we’re nearly there. Now write a blog post, copy the URL, paste it to the Embedded Media field, paste to the FeedAPI URL field, and we’re done. Wait, so why do we need to paste it twice? Well, because there are two fields with the same value.

So let’s hide one and copy the value from the exposed one. We whipped up a glue module that hides the FeedAPI fields if an EMField is present and populates the hidden field with the EMField value on validation. The module is very simple and has no configuration. It just hides all FeedAPI fields if an emfield is present. For convenience sake, it ships with FeedAPI Comments, its name is “FeedAPI Comments — Emfield integration”.

Behind the Scenes

The most exciting part for me is how simple it is to write comment parser integrations for new sites. The FeedAPI Comments parser is an API for defining page scrapers. For example the information for scraping the comments from consists of a series of XPath definitions:

// start site-specific data
$post_vars = ''; // if the comments are available after a POST request

// Currently one path can contain one placeholder, no more (imagine, one placeholder - running value  - one more loop inside loops!
$xpath_comments = array(

// No placeholder support in data members yet
$xpath_data_members = array(
  'title' => '',
  'body' => '//td[@class=\'Said\' and position()=2]/p/text()[1]',
  'author' => '/html/body/tr/td[2]/h4/a',
  'author_picture' => '/html/body/tr/td[1]/a/img/@src',
  'date' => '//p/small',
  'original_url' => '',
$placeholders = array(
  '%place' => array(1, UNLIMITED),

// To decide if the given video feed is okay for this handler.
// preg_match, so perl-style please
$url_checker_regexp = '/\/photos\//';

If you know XPath, you can write a comment scraper definition for a new site within an hour. This API is a pretty effective concept and could be easily generalized to any scraping job — not just comments.

Props to the great work done by the maintainers of emfield and to Aron Novak, who wrote FeedAPI Comments module.

If you would like to try comment aggregation yourself, be sure to you use FeedAPI Comments 1.0 and the latest development version of FeedAPI.

What we're doing.