Getting diffs for Puppet catalogs

by R.I. Pienaar | Nov 14, 2010 | Uncategorized

Puppet compiles its manifests into a catalog, the catalog is derived from your code and is something that can be executed on your node.

This model is very different from other configuration management system that tend to execute top down and just run through the instructions in a very traditional manner.

Having a compiled artifact has many advantages most of which aren’t really exposed to users today, I have a lot of ideas on how I would like to use the catalog – and the graph it contains. The first idea is to be able to compare them and identify changes between versions of your code.

For this discussion I’ll start with the code below:

class one {
    file{"/tmp/test": content => "foo" }
}
 
class two {
    include one
 
    file{"/tmp/test1": content => "foo";
 
         "/tmp/test2": content => "foo";
    }
}
 
include two

When I run it I get 3 files:

-rw-r--r-- 1 root root 3 Nov 14 11:32 /tmp/test
-rw-r--r-- 1 root root 3 Nov 14 11:32 /tmp/test1
-rw-r--r-- 1 root root 3 Nov 14 11:31 /tmp/test2

Being able to diff the catalog has a lot of potential. Often when you look at a diff of code it’s hard to know what the end result would be, especially if you use inheritance heavily or if your code relies on external data like from extlookup. Since the puppet master now supports compiling catalogs and spitting them out to STDOUT you also have the possibility to compile machine catalogs on a staging master and compare it against the production catalog without any risk.

The other use case could be during major version upgrades where you wish to validate the next release of Puppet will behave the same way as the old one. We’ve had problems in the past where 0.24.x would evaluate templates differently from later versions and you get unexpected changes being rolled out to your machines.

Lets make a change to our code above, here’s the diff of our change:

--- test.pp     2010-11-14 11:35:57.000000000 +0000
+++ test2.pp    2010-11-14 11:36:06.000000000 +0000
@@ -5,6 +5,8 @@
 class two {
     include one
 
+    File{ mode => 400 }
+
     file{"/tmp/test1": content => "foo";
 
          "/tmp/test2": content => "foo";

This is the kind of thing you’ll see in mail if you have your SCM set up to mail diffs or while sitting in a change control meeting. The change looks simple enough you want to just change the mode of /tmp/test1 and /tmp/test2 to 400 rather than the default.

When you run this code though you’ll see that /tmp/test also change! This is because setting defaults applies to included classes too, and this is exactly the kind of situation that is very hard to pick up from diffs and to be able to guess the full impact of the change.

My diff tool will have shown you this (format slightly edited):

Resource counts:
        Old: 516
        New: 516
 
Catalogs contain the same resources by resource title
 
 
Individual Resource differences:
Old Resource:
        file{"/tmp/test": content => acbd18db4cc2f85cedef654fccc4a4d8 }
New Resource:
        file{"/tmp/test": mode => 400, content => acbd18db4cc2f85cedef654fccc4a4d8 }
 
Old Resource:
        file{"/tmp/test1": content => acbd18db4cc2f85cedef654fccc4a4d8 }
New Resource:
        file{"/tmp/test1": mode => 400, content => acbd18db4cc2f85cedef654fccc4a4d8 }
 
Old Resource:
        file{"/tmp/test2": content => acbd18db4cc2f85cedef654fccc4a4d8 }
New Resource:
        file{"/tmp/test2": mode => 400, content => acbd18db4cc2f85cedef654fccc4a4d8 }

Here you can clearly see all 3 files will be changed and not just two. With this information you’d be much better off in your change control meeting than before.

The diff tool works in a bit of a round about manner and I hope to improve the usage a bit in the near future. First you need to dump the catalogs into a format unique to this tool set and finally you can diff this intermediate format. The reason for this is that you can compare catalogs from different versions of puppet code so you need to go via an intermediate format.

There’s one thing worth noting. I initially wrote it to help with a migration from 0.24.8 to 0.25.x or even 2.6.x and in my initial tests this seemed fine but on more extensive testing with bigger catalogs I noticed a number of strange things in the 0.24.x catalog format. First it doesn’t contain all the properties for Defined Types and 2nd it sets a whole lot of extra properties on resources filling in blanks left by the user.

What this means is that if you diff a 0.24.x catalog vs the same code on newer versions you’ll likely see it complain that all your defined type resources are missing from the 0.24 catalog and you might also get some false positives on resource diffs. I can’t do much about the missing resources but what I can do is clear up the false positives, I already handle the ones in my manifests but there are no doubt more if you let me know of them I’ll see about working around them too.

The code for this can be found in my GitHub account. It’s still a bit of a work in progress as I haven’t actually done my migration yet so subscribe to the repo there’s likely to be frequent changes still.