Blog: Aggregators

How To Create A Mapper

For those of you who after reading my last post on the Feed Element Mapper are now interested in the actual works of the module, below is a step by step guide on how to build your own mapper.

But first, three things needed to happen before we were able to create the map-any-element-to-CCK feature:

• Jess from WETA poked me about it for their TV and Radio feeds at the Drupal Meetup in DC on Tuesday :)
• FeedAPI's parser_simplepie now passes on the raw feed item, and with that anything it detects on the feed, on feed_item->options->raw
• Feed Element Mapper offers a generic CCK mapper

From those three, the one I would like to explain a bit better is the third because it shows how easy it is to write your own mapper so you can map any feed item element identified by FeedAPI to virtually any property of a node. Let me explain how I built this mapper:

1)
This is the empty implementation of hook_feedapi_mapper(). It's just sitting there and doing nothing. In this case, it' the implementation for the content module, the hook starts with content_.

function content_feedapi_mapper($op, $node, $field_name, 
				$feed_element = array(), $sub_field = '') {

}

• $op tells us which operation is being executed ('describe', 'list,' or 'map').
• $node is the node object we're operating on.
• $field_name is the name of the field on the node we are being called with (e. g. 'field_severity' for $node->field_severity, or 'taxonomy' for $node->taxonomy).
• $feed_element is only set on $op == 'map' and contains the exact element that has been mapped to the given field. This element can be a string, a number or an array of string or numbers.
• $sub_field is advanced stuff - it contains a value only if you offer more than one mapping target on $op == 'list', which we won't do here (e. g. a taxonomy mapper offering a mapping for each vocabulary).

Use the Feed Element Mapper to Map Any RSS Tag to Taxonomy or CCK Fields

The first time I talked here about the Feed Element Mapper, it was still quite basic - it just mapped feed elements recognized by FeedAPI to taxonomy terms. That was already pretty cool, but what about mapping to CCK fields, and what about mapping any tag - even custom ones - to CCK fields? 

Well, the latest developer version of FeedAPI (Jan 23rd), together with Feed Element Mapper 1.0 beta 3, can do this. It offers a straight forward form manageable also for the not so experienced site builder. I recorded this screencast to show you how this plays out on the UI:

If you would like to try this, take care that you do the following:

• Use FeedAPI 1.0
• Use Feed Element Mapper >= 1.0 beta 3
• Configure your feeds to use parser_simplepie as parser.

FeedAPI Now Maps Feed Elements to Node Fields

Last week I did some cleaning up of the soon-to-come FeedAPI beta release. I got fired up and decided to address one of the more exciting feature requests for Drupal aggregators - a feed element mapper. This feature will make it possible to freely map elements on feed items to fields on nodes that are created from these feed items.

The feed element mapper is an add on module for FeedAPI. It's currently in a proof of concept state and therefore lives in my sandbox. This screencast shows a demo of the straightforward mapping process.

• Feature request on FeedAPI
• Feed element mapper (sandbox)

For the developers

At the moment it's only possible to map to taxonomy, although writing your own mapper should be easy. This example of the taxonomy mapper shows how you can define a feed element mapper for your node field. First your mapping function is asked which fields it exposes ($op == 'list'). Then, when the user defines a feed element for one of the fields you exposed, the function is called again with the mapped elements and the node to stick them onto ($op == 'map').

function feedapi_mapper_map_taxonomy($op = 'map', $node, $feed_element = array(), $sub_field = '') {
  if ($op == 'list') {
    if ($vocabularies = taxonomy_get_vocabularies($node->type)) {
      foreach ($vocabularies as $v) {
        $sub_fields[$v->vid] = $v->name;
      }
      return $sub_fields;
    }
    return FALSE;
  }
  else if ($op == 'map') {
    if (is_string($feed_element)) {
      $feed_element = array($feed_element);
    }
    if (is_array($feed_element)) {
      if (!is_array($node->taxonomy)) {
        $node->taxonomy = array();
      }
      $node->taxonomy = array_merge($node->taxonomy, _feedapi_mapper_create_terms($feed_element, $sub_field));
    }
    return $node;
  }
}

Beta Version of the FeedAPI Module Ready for Use

This summer I created an aggregation API for Drupal as part of Google’s Summer of Code project. The module I wrote - called FeedAPI – has functionalities that will be of interest to both end users and developers. The module serves as a general purpose aggregation module that’s able to behave like the classic core aggregator or like Leech (among others), and it allows users to create feeds and view items easily. However the main concept behind the project will be of most interest to developers. The module allows feeds to be processed by various parsers and processors, and the API supports all types of feeds and feed item representation.

The FeedAPI module is in beta version, and with the exception of some bugs filed in the Drupal issue queue that I need to address, it’s ready to be used. To see what the module can do, I recommend that you watch the screencast and visit the live demo site.

You can find the latest package of the module here. I encourage developers to check out the developers’ guide and want to thank those in advance who test it out (and special applause for those who give feedback!).

Pop Quiz: If you watch the screencast attentively, you’ll see one already reported bug. Can you spot it?

Hot Tags and the Tags Related to Them

Think about the news you track and the issues you need to stay on top of on a daily basis. For me, one of those topics is social networking. There are often thousands of stories about social networking published every day with perhaps hundreds of different angles to them. Maybe somewhere in those thousands of stories and hundreds of angles is an emerging trend – major research on its impact on job hunting, a new technology or service that the blogosphere is drumming up, or the use of these networks by a new segment of the population. But how can I spot the trends in all that news?

Well, graphing the news can help. When you look at the incidence of a word in the news and can see the rise and fall of its use on a day-to-day basis, you can tell if a subject is gaining momentum in the media. However, seeing the rise and fall of a word’s incidence does nothing to help determine why it’s getting more or less attention. To figure that out you need more information. One way to do this is to look at words related to the hot topics in the media.

This graph shows the mentions of MySpace in the media from July 8 to July 30 (data from Trendio.com*). You can see that there is a slight spike toward the end of the chart and that mentions for MySpace peak at 164 on July 27. But why? 

Major Progress for the FeedAPI Module

Aron Novak has spent the past few weeks hard at work developing a new and improved aggregator module for Drupal as part of Google’s Summer of Code. The FeedAPI module takes on the common tasks of the most popular existing aggregator modules and has an API to extend it or tweak it to your needs. The hope is for this module to become the foundation for all future aggregator modules in Drupal, and allow other modules to just plug in to it to add new functionalities.

This module is in a hot phase of development right now, and all input is highly appreciated. So please check it out and give Aron your input.