Drush Make Files for Production Drupal sites
Deployment with a safety net
We now write Drush Make files for even our smallest one-off builds. This discipline saves us time and makes us more efficient as a team by knowing what code actually powers a site. It also helps us transition projects to our partners or to the shops that will be supporting the sites we build, helping us more quickly cut the long tail of a build. To give a sense of just how much this can help internal processes, I'll explain how I was robbed of three days this week by jumping onto a project we worked on before we formalized using Drush Make in our build process.
This week we started sprinting on a custom Open Atrium build for Teachers Without Borders, an organization with a somewhat obvious multinational reach that wanted us to extend Open Atrium in a few interesting ways. They needed Open Atrium to be an identity hub, be localized into multiple languages, and have more social features like following. We'd already done some work for Teachers Without Borders deploying a set of Managing News sites, including one localized into Chinese, and an Open Atrium hub site with the OpenId Simple Sign On stack.
When we originally deployed these sites, we didn't have a good way of tracking what had been added or changed from the standard Open Atrium and Managing News platforms. Each of these platforms had a number of extra modules deployed, mostly to power the authentication infrastructure, and a patch or two to some modules. As someone who didn't work at all on either of those projects, it made getting into these two sites, as well as fixing bugs on them, a very confusing experience.
After an initial look at the code base, I decided that the best way to capture what was there and prevent others from suffering from the same confusion was to create make files for both projects. This meant examining each production platform and trying to distill what versions were running with what patches. Luckily Will, who had worked on this Open Atrium deployment, was able to help with the Open Atrium customization reconstruction. For Managing News I was a bit more on my own, though Alex was able to help me with some questions. Still, the process of reconstructing a customized platform that you didn't work on is error prone. If it can be avoided, it should be.
The process I adopted for both of these sites was to fork the install profile into a new repository and start making changes. An "include" directive was recently added to Drush Make that allows you to include the contents of a remote make file in one that you're creating. Using this I was able to delete nearly the entire contents of managingnews.make and replace it with an 'include' directive that pulled in the entire stable Managing News make file:
includes[managingnews] = http://drupalcode.org/viewvc/drupal/contributions/profiles/managingnews/managingnews.make?revision=1.1.2.85&view=co&pathrev=DRUPAL-6--1Once I'd done this I could add details to the make file about what was custom for this site. For example, I could add modules not normally a part of Managing News to the project, or I could add patches to modules defined in the stable make file. I could also switch the version of a module I wanted to use, such as switching from the public release of the mn_core module to a checkout from github.
projects[mn_core][type] = "module"
projects[mn_core][download][type] = "git"
projects[mn_core][download][url] = "git://github.com/developmentseed/mn_core.git"
projects[mn_core][download][tag] = "drupal-6--1-0-beta20"Using a make file to track the customizations in a project leaves a record of which version of Open Atrium or Managing News you started from and exactly what is different. When a new developer comes onto the project, there is a definitive list of where the customizations start and how many exist. New developers don't need to play a game of guess and check, instead they have a definitive record to look at. In these situations, the make file is your safety net.
What’s the effort and time
What’s the effort and time required to create and maintain the make file?
I imagine that the make file approach would be more suitable for larger projects that have their own hosting environment as opposed to a multi-site project with shared module instances and Drupal core.
For a shop that hosts and manages small to mid-size sites for their clients, the two clear benefits I see from make files are: the ability to quickly hand over the project or move it to it’s own hosting environment – and have an immediate understanding of what the project components are.
I’d be willing to include the make file approach in our work-flow if the process to creating the make file was as simple as drush makefile filename.make and perhaps a few tweaks thereafter for the repo include urls of specific upstream components.
Ignorance is bliss?
I’ve been mulling this one over for a few days. Great food for thought. I think the benefit and risk of this approach is that you give up a lot of (perceived?) control. For example, lets say that you finally take the time to pay attention to your Updates report and you update 3 contrib modules to their latest releases. You update the make file and test your site and you release.
Then, bug reports start coming in. Its likely that the new code has broken something. One would usually turn to your own repository and generate a diff which includes all changed code on that day. But with this Make system, you can’t. The best you can do is generate a diff for each Contrib module. And sometimes the real problem was in your own custom code which changed in the same period. Thats another repository to check.
Jeff articulates lots of benefits for this approach. I think it will appeal to the many people who aren’t version control gurus because all the core/contrib tracking goes away. It won’t appeal to control freaks. They need to have a complete diffable history of all changes on their site.
The approach of using a make
The approach of using a make file to track deployed versions of various projects is certainly an alternative to using a single repository to track all the projects that make up a site. Depending on how close to the edge of development you’re running it may not make sense. The make file approach certainly assumes that you’re site it trying to track upstream repositories and release – which may not always be true, but for most sites it is.
With regard to generating a diff – this should still be easy, though not as quick as if you’ve already committed everything to a single SVN repository. Assuming you’re make file is under version control, you’d simple check out the last release tag, build that, checkout the current release, build it and diff them. You’d get exactly the same diff, although perhaps 5 minutes later.
Using a project repository like this however you’re intermingling you’re customizations with upstream code. With the this approach, I’ve had fixes to contrib modules blown away when the repository is synced with it’s upstream source. Sure, this is generally caused by a little bit of carelessness, but with the make file approach it can’t happen. If a patch won’t apply you’ll get an error.
What is really different about a make file is that it is a concise definition of the site that can be easily understood quickly. Even if I had come to the sites I describe in the the blog post and everything was in subversion, I’d still have to go module by module to determine what versions were running with which patches, and be pretty much left guessing about CVS checkouts. There are certainly ways to document your way out of this problem ahead of time, but they’ll end up containing the exact same information as a make file with the added requirement that the documentation is a secondary source that needs to be updated once you’ve updated the repository with a new version of code. Using a make file makes this into a single action.
with a safety net
Just noticed the tagline ‘deployment with a safety net’. This sounds like develoment (not deployment) without a safety net. What am I missing?
As for “deployment” in the
As for “deployment” in the sub-title, It was simply was the situation in which I ran into this issue. I had deployed sites that needed updates, and the site contained code from 4+ different repositories each, and had some “natural growth”.
While I can see how this approach could be competition for single project repository approach. I certainly see these two approaches useful, and both as safety nets.
For these two sites, one that tracks Managing News closely and the other that tracks Open Atrium closely, the make file has the really tremendous advantages of not making me do work twice; once in OA/MN and again on sites like these.
What may be the missing piece for why this is about deployment and not development, is how we’re packaging and hosting. On these builds we have a script that makes a full package as a tarball, ready to be uploaded to our Aegir servers. The Open Atrium repository has a script `rebuild.sh` that is a good example of this.
So the workflow for fixing a bug found in production ends up being something like; 1) checkout the make file and build the site, 2) fix the bug & commit changes the appropriate repositories 3) build a tarball and upload it to the server. 4) use Aegir to migrate the site to the new codebase.
nice workflow
Love that production bug fix workflow. Will be interesting to see how many other shops adopt this strategy.
Looking forward to your talk at CPH.
Will you be releasing this code back out?
I’m working with a client who just deployed Jive, and they like the follow feature for instance, but I’ve been showing them OA, and if it had this feature for instance, I think they would consider switching…
include directive
Thanks for posting this Jeff.
I use drush_make for all my projects now but I hadn’t noticed the include directive before and sounds like just what I need.
Pulling updated code
So this all makes perfect sense, except I am wondering one thing:
When you want to update your project, e.g. Updating to Features Beta10, do you just run drush dl on that one project (and if necessary update the make file) or do you periodically run the entire make file?
Or if you are really bold and using aegir I suppose you could configure a script to periodically rebuild using the make file and then migrate the site. Automatic updates of sorts?
We generally have a
We generally have a “rebuild.sh” script in the project that rebuilds makes rebuilding the entire make file a single (and simple) command. There is a script like this that ships with Open Atrium, however it’s a bit more complex than the average site file needs.
I haven’t ventured in to Aegir based rebuilding yet, but it’s absolutely on my list.
Alright I can get organised
Alright I can get organised now
very cool
We usually keep patches in a directory with a README for additional details and rely on just looking at the modules directory for such things. However, coming into a new project I can feel your pain. I probably would have gained a couple of weeks coming into the the middle of a project recently had there been a make file giving me a definitive list of what was involved already. I’m going to give this a shot on my next project.
Make files replaces single repository approach?
So these make files replaces the tracking of the project as a unit that many, including me, uses version control like Git for?
You version control this make file in one repo and the individual modules in their repos and have no repo where everything is gathered?
What do you do when you need to change/patch a module – do you start changing the module in a site created with the make-file without any version control or do you manually swap that module out for a version controlled module until any fixes has been committed or patches has been created?
What about version control of patches – do you do any such thing?
What would you say is the specific advantage of this compared to eg. a single git with submodules or subtrees?
Yes, this is a bit different
Yes, this is a bit different than the ‘entire project in one repository’ approach that many people do. The advantages are; 1) that you’re not double tracking a project that also has it’s own canonical repository, and 2) that patches to such projects are distinct, and not buried in a git log which may also contains updates from the canonical repo.
When we need to start patching a module from drupal.org, we will generally switch the make file to retrieve a CVS checkout, as opposed to pulling a packaged release. Then we’ll use CVS to generate the patch.
Patches that we write for drupal.org modules are submitted to the issue queue, and we refer to the patch’s location on drupal.org in the make file. This has made us much better contributors to other people projects as it makes being involved in the issue queue a normal part of development, and it encourages us to only patch contrib modules where it’s likely that the patch will be accepted. When a patch gets a review, we make changes, upload a newer version of the patch to drupal.org, and update our make file.
It’s certainly a lot different than what we used to do, but this process has given us a lot more flexibility, and just making it easier to stay involved in the drupal.org queues make it totally worth it.
Also, I’ll be giving a talk about this topic at Drupalcon Copenhagen which would be interesting if you’re going to be in Copenhagen & want more details about the nitty gritty of this workflow.
misunderstood?
I don’t think they mean to say that this method ‘replaces’ version control, it simply is a nice structure to define a project so that when new people come onto the project, they have a well ‘documented’ structure to help them figure out what all has happened thus far.