Importing and Aggregating Stuff with Feeds


3 min read

Nearly three months ago we released the Feeds module. When we introduced it, I explained that Feeds has a fundamental dual nature as both an import framework and an aggregator. When you think about it, aggregation is nothing other than a scheduled import, so why would you build two different infrastructures for virtually the same functionality?

I’d like to illustrate this point with three examples that show how Feeds can be used to aggregate RSS and import users or nodes from CSV. If you would like to follow these examples, all you need to do is install the latest version of Drupal 6 and the latest version of Feeds 1.0.

1. Aggregate RSS or Atom

This video shows how an RSS feed like the one on our blog can be aggregated using Feeds. It further explains how to adjust the default settings to capture categories on the feed as taxonomy.

2. Import Nodes

This video shows how nodes can be imported into Drupal and how to adjust the default settings to populate CCK fields.

3. Import Users

This video shows how users can be imported into Drupal. It takes a look at how default settings need to be adjusted for different CSV files.

Looking around

If you are looking for import solutions in a migration context, there are alternatives to Feeds such as Migrate, Table wizard with CSV import support, User import, and Node Import.

Migrate and Table wizard enjoy very active development and are used for complex migration tasks. You want to use them if your data needs cleaning and reorganizing before an actual import. I could see Feeds helping these modules by providing a pluggable import functionality (which would be as easy as writing a Feeds processor that populates database tables). To what extent this makes sense is a discussion that I’d be happy to have with the maintainers.

User import is a great module if you are looking for more advanced functionality around importing users (for instance notifying the imported users by email).

From all the modules that I’ve mentioned, I see the most overlap with Node import. Both Feeds and Node import aim to offer full blown import functionality for nodes.

In comparison to the modules mentioned here, Feeds’ philosophy is to allow recurring action and pluggability. Feeds does not make any assumptions on whether an import is being used at build time or throughout the life time of a site. It provides a common architecture on which the three stages of importing stuff (fetching, parsing, processing) rest and it allows the site builder to configure these three stages to fit the use case at hand.

Looking forward

In the last two months several developers have gotten onto the Feeds train and have contributed valuable additions and fixes. While we use Feeds in production, we consider it in progress and I encourage interested developers to join in. Here are some of the issues that are high on the priority list:

What we're doing.