In the last couple months we’ve been driving efforts to bring Drupal’s core aggregator up to speed so that we could use it to replace FeedAPI in Drupal 7. After trying to pull through some of the more extensive changes, I have to recognize that in this case the advantages of Drupal core work are not enough to outweigh the disadvantages.

A year ago when “we decided to help improve Drupal’s core aggregator”:http://www.developmentseed.org/blog/2008/jul/18/why-heck-new-aggregator-drupal-7 to a point where it could replace FeedAPI, my assumption was that the additional effort of working in core would be more than compensated by the important reviews and input we would get from the Drupal community.

Indeed, we’ve received some great help and I’d like to thank everybody involved. But it became very obvious that we could not gain the momentum I was hoping for. Patches like #303930: pluggable architecture and #293318: feeds as nodes, which I consider central to the effort, were only reviewed in depth by a few and overall the patches moved slowly.

I don’t think this lack of support can be blamed on anybody in particular, but I do think it shows that there is some discrepancy between what we are trying to do with aggregation in Drupal — namely building a flexible, extensible aggregation API — and what is reasonably possible within Drupal core.

At the same time we started investing in core aggregator, our demands in aggregation escalated. With every new client project we undertook, we contributed critical changes to FeedAPI and launched many new plugins for it. These contributions made the existing deficiencies of FeedAPI clearer and the gap that needed to be closed with its successor — Drupal 7’s aggregator — wider. We are now at a point where we cannot guarantee anymore that the slow moving and remote solution of a “Great New Aggregator for Drupal 7” will address our needs. Piling on extensive changes without simultaneous use or proper review is not the recipe for a solid module.

Therefore, we need to adjust our goals. Instead of replacing FeedAPI with Aggregator in Drupal 7, we will aim to develop a successor for FeedAPI in contrib. This approach will give us a chance to let the base of the FeedAPI module mature and give us more leeway for new features and adjustments.

And there is much to do. In the last year, FeedAPI has proven its potential as a universal importing platform, and I think this aspect of it will only become more important. We will need a cleaner and more powerful API, fast and flexible storage for simple feed items, better support for large feeds, and better performance monitoring.

I will share more concrete ideas and ask for your feedback as these ideas are taking shape. What is for sure is that there will be an upgrade path for FeedAPI into Drupal 7 and — of course — an upgrade path for Aggregator into Drupal 7.

This decision was not easy, but I believe that aggregation in Drupal needs — at least — another round in contrib, and I’m excited about the opportunities lying ahead.