Puppet Environments

by R.I. Pienaar | Oct 10, 2009 | Code

Puppet supports putting nodes in environments, this maps cleanly to your development, qa and production life cycles and it’s a way to hand out different code to different nodes. Before reading this post you should probably read the Puppet Wiki page first for background.

I’ll present here a simple layout that is both a powerful tool for isolating code changes while developing combined with minimizing the headaches with maintaining multiple branches of the same module for an extended period of time.

As with the other posts I did on puppet recently the focus here is simplicity, this will work for small sites and will hopefully give you enough inspiration to successfully model larger sites using these techniques.

To really use environments well I strongly suggest you use modules, I’ve blogged about them before, if you read this post and think “hey! that’s great” and don’t yet use modules, go fix that first.

The puppetmaster tries to find modules using it’s modulepath setting, typically something like /etc/puppet/modules.  You usually just set this value once in your puppet.conf and that’s it, all done. Environments expand on this and give you the ability to set different settings for different environments, below a simple example.

[puppetmasterd]
   modulepath = /etc/puppet/manifests/modules
 
[development]
   modulepath = /etc/puppet/manifests/development/modules

This is your basic multi environment setup, if you run a client with the development environment set modules will be served from an alternative path.

This is already a huge win, if you use something like below:

class ntp {
   file{"/etc/ntpd.conf":
      source => ["puppet:///ntp/ntpd.conf.${environment}", 
                 "puppet:///ntp/ntpd.conf"]
   }
}
 
node dev1 {
   include ntp
}

If you put the ntp module in both production and development, the right one will be chosen depending on the nodes environment setting. The file ntp.conf will also be served from the right modules files directory. You could now safely make changes to the development one without affecting the rest of the machines.

The major drawback here is that if you have say 30 modules, you need to duplicate the lot into development if you wish to use them there, this is not great at all as it quickly becomes a maintenance nightmare. If you made a change to ntp module in development, you need to remember to go and do the same change in all your other environments. In cases where you give each sysadmin or developer their own environment this rapidly becomes unmaintainable.

The modulepath option has a trick up it’s sleeve though, you can specify several search paths just like the normal Unix PATH environment variable, we can rework our config like this.

[puppetmasterd]
   modulepath = /etc/puppet/manifests/common/modules
 
[development]
   modulepath = /etc/.../development/modules:/etc/.../common/modules
 
[production]
   modulepath = /etc/.../production/modules:/etc/.../common/modules
 
[johndoe]
   modulepath = /etc/.../johndoe/modules:/etc/.../common/modules

Here we’ve added a few more environments, even one for a developer called John Doe. The big thing we did though is create /etc/puppet/manifests/common/modules as a 2nd search path.

production
`-- modules
development
`-- modules
common
`-- modules
    `-- ntp
        `-- manifests
            `-- init.pp

The idea here is that you’d strive to keep all your modules in the common directory as their stable location but if you wished to do development on your ntp module that you wish to test, you just copy the ntp module into either development or johndoe directories, with that done only machines in those environments will get the new code.

This works especially well if you use version control, I use SVN and treat common as my trunk, whenever I need to work on a module I simply branch it into development, do my work there and when ready to release I merge my changes back to common and get rid of the branched copy so all environments gets the same version.

With this you can easily come up with a simple release testing strategy, do the weeks code in development and test locally, then put the changes you wish to release to production into your QA environment and do tests. Finally when you’re ready merge all the changes into common and just get rid of the development and QA copies, all environments are now running the same code ready for the next cycle.

This gives you the ability to create branches, do testing, isolate developers from each other – just put their desktops in individual environments – and lots of other nifty things but in the end you have only one copy of the code for your stable released versions.

I hope you find this useful, I’ll reiterate that this is a simple setup, you should use this information and come up with a solution that will work well for your use case and teams.