Development Seed

Originally announced a few months ago alongside the 0.3 release, the Aegir project is in the process of finalizing its migration to git for our revision control needs. For the last few releases, several of our developers have actually been using git for development, and this week we are simply making our preference official. As of today, the official source for the packages that make up the Aegir Hosting System will be git.aegirproject.org. While we do not have an official website up at AegirProject.org, rest assured that is coming too.

I'd like to go through some of the reasons and benefits of this change.

Being able to make sweeping changes without losing history

Because our API is evolving so rapidly, we are not averse to major refactoring of the code base. Because of the inability of CVS to rename files and directories, this has been a thorn in our sides when performing these refactoring tasks. Some of you might remember the period in the Drupal 5.x development cycle when the modules were moved to their own directories instead of just being .module files. It was chaos for awhile.

We are standing at the precipice of another refactoring of this sort, and it is very likely going to be a case where actually getting the changes committed the right way will take longer than making the changes themselves. We would have to compromise and remove the files being renamed and commit them anew, losing several years of important development history.

Smarter collaboration between developers

A number of our core developers are already making use of 'feature branches' for sweeping changes. Because we will all have our own repositories to commit code to before merging them in, it will allow us to collaborate on new features far more easily than shuttling patches between each other.

This becomes especially painful with CVS because new features tend to mean new directories and files that are created, so actually generating the necessary patches becomes an exercise in working around CVS' limitations and it becomes very easy to lose entire files from patches.

Our project is very very distributed (over three continents and time zones!), and it only makes sense that we need to use a system that enables us to work together as effectively as possible.

Drush Make means the changeover is nearly seamless

One of the primary reasons Drush Make has become a requirement for all new releases of Aegir is that it allows us to sidestep the entire packaging problem for install profiles. The largest part of our installation documentation in the past has been instructions to successfully assemble all the various components in the system in the right place. Because of this we have been incredibly careful about adding any new dependencies to the system, even if they are well tested and widely used (see: views etc.).

As of our latest alpha release, we now ship a makefile with the backend and provide a few additional commands. Once you have the backend, drush, and drush make installed, you are able to simply type:

This will download all the necessary components for the frontend from all the various sources and allow you to continue with installation. When a new release of Aegir is available, you will simply need to update the provision backend and then run:

This will download and upgrade your frontend to the release meant for the backend you have installed.

Because drush make supports fetching components from git, the only change that will affect our end users is that they will have to download the provision script from somewhere that isn't Drupal.org.

Able to structure our repositories in a way that makes sense to us

The Aegir project is made up of an extension to drush, an install profile, a module, and a simplified administration theme. All of these parts work together to form the system, and each release of Aegir means releasing all of these components. While there is nothing wrong with this on the backend, on the frontend all the components are designed to work only in the presence of each other. Because Drupal.org does not allow modules or themes to be checked into the profiles folder, we are required to have separate Drupal projects for each of these components.

Separate projects means that we need to commit to multiple repositories, tag multiple times, manage multiple issue queues, and write multiple release nodes for every release of the system. Aside from this administrative overhead, it also makes it more troublesome to develop each of the pieces, since you have to CVS update 4 different directories, write different commit logs, test multiple patches for a single change, and so forth.

By moving to git we will finally be able to structure our code in a way that makes our lives simpler, streamlines development, and reduces the margin for error when making releases.

These limitations on Drupal.org have a very good reason for existing. It is felt that allowing modules to be committed to the profiles folder would allow people to fork modules into the profiles folder, and therefore become a maintenance nightmare. This is especially true in the absence of proper packaging of install profiles.

Managing our releases the way we need to

As a project we have a very definite idea of where we are headed and part of the implementation of this roadmap is evident in our versioning. Each of our major releases so far has been 0.1 - 0.4. We do not want to release a 1.0 until we have a stable API that will allow contributed modules to work for the entire lifetime of the 1.x cycle. Who wants to rewrite their backend provisioning code every six months?

Our problem comes in because our release versioning requirements are at odds with the limitations of the Drupal packaging system. We are currently doing all our development on HEAD, and when a new release is ready we tag it as 0.x and make the release, but if we find an issue in a release we are actually physically unable to make a 0.x.1 bugfix release because of the enforced tagging requirements on CVS. Anyone who has worked with branches in CVS knows how difficult maintaining those stable releases would be if they were branches. Even with the addition of branching, we would need to alter our roadmap every time we make a bug fix release, which limits our ability to maintain multiple releases.

I should add that in no way does this mean that the CVS requirements need to change. We are very very aware that we are an edge case that goes against the grain of how normal Drupal projects are managed. These choices are our own, but they are important to us.

Branching without being forced to follow a Drupal branch

The provision script, which acts as Aegir's backend, is not specific to any single Drupal release. In fact it operates on Drupal 5, Drupal 6, and (somewhat) Drupal 7 releases, and it runs into the fact that the current release system on Drupal.org simply isn't designed for something that bucks the rules in the way that it does.

Drush is an example of this problem, in that its release tags are DRUPAL-6--2-1, but after the node is created the release node is modified by someone with the correct permissions to create a tarball called drush-All-Versions-2.1.tar.gz.

Originally we were aiming to call the 0.3 release 6.x-0.2 , since it was an exact port of the Drupal 5 release with no extra features. But during development we found that we needed to make some changes to provision, and we had already been using the DRUPAL-6-0.2 tags for the previous release as provision supported Drupal up to 6.x.

Since we were unable to make a 0.2.1 release, we decided to name our next release 0.3 instead, and since then the Drupal major version for all our packages have become irrelevant. The use of drush make will also make the version of Drupal the front end is running on irrelevant. It will simply fetch the correct Drupal release to run the specific version of the front end.

In closing

We are taking over hosting of our version control not because we dislike Drupal.org, but because we are simply accepting that our requirements are slightly at odds with the assumptions that work for 99% of Drupal projects. Drupal.org is an amazing resource, however at this time the current infrastructure does not meet the needs of the Aegir project.

This change in version control allows us to be more efficient and serve our end users better, without requiring us to compromise on the way we choose to run our project. In the same way that sites outgrow shared hosting, the Aegir project has outgrown shared project management provided by Drupal.org.

Follow @aegirproject to keep up to date with the latest developments in the Aegir project.