Since shutting down my business I now run a small 25 node network with no Puppet Masters and I do not schedule regular Puppet runs – I run them just when needed.
Till now I’ve just done puppet runs via MCollective, basically I’d edit some puppet files and after comitting them just send off a puppet run with mcollective, supplying filters by hand so I only trigger runs on the appropriate nodes.
I started looking into git commit hooks to see if I can streamline this. I could of course just trigger a run on all nodes after a commit, there is no problem with capacity of masters etc to worry about. This is not very elegant so I thought I’d write something to parse my git push and trigger runs on just the right machines.
I’ll show a simplified version of the code here, the full version of the post-receive hook can be found here. I’ve removed the parse_hiera, parse_node and parse_modules functions from this but you can find them in the code linked to. To use this code you will need MCollective 2.0.0 that is due in a few days.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
#!/usr/bin/env ruby require 'rubygems' require 'grit' require 'mcollective' include MCollective::RPC @matched_modules = [] @matched_nodes = [] @matched_facts = [] # read each git ref in the push and process them while msg = gets old_sha, new_sha, ref = msg.split(' ', 3) repo = Grit::Repo.new(File.join(File.dirname(__FILE__), '..')) commit = repo.commit(new_sha) case ref when %r{^refs/heads/(.*)$} branch = $~[1] if branch == "master" puts "Commit on #{branch}" commit.diffs.each do |diff| puts " %s" % diff.b_path # parse the paths and save them to the @matched_* arrays # these functions are in the full code paste linked to above case diff.b_path when /^hieradb/ parse_hiera(diff.b_path) when /^nodes/ parse_node(diff.b_path) when /^common\/modules/ parse_modules(diff.b_path) else puts "ERROR: Do not know how to parse #{diff.b_path}" end end else puts "Commit on non master branch #{branch} ignoring" end end end unless @matched_modules.empty? && @matched_nodes.empty? && @matched_facts.empty? puppet = rpcclient("puppetd") nodes = [] compound_filter = [] nodes << @matched_nodes # if classes or facts are found then do a discover unless @matched_modules.empty? && @matched_facts.empty? compound_filter << @matched_modules << @matched_facts puppet.comound_filter compound_filter.flatten.uniq.join(" or ") nodes << puppet.discover end if nodes.flatten.uniq.empty? puts "No nodes discovered via mcollective or in commits" exit end # use new mc 2.0.0 pluggable discovery to supply node list # thats a combination of data discovered on the network and file named puppet.discover :nodes => nodes.flatten.uniq puts puts "Files matched classes: %s" % @matched_modules.join(", ") unless @matched_modules.empty? puts "Files matched nodes: %s" % @matched_nodes.join(", ") unless @matched_nodes.empty? puts "Files matched facts: %s" % @matched_facts.join(", ") unless @matched_facts.empty? puts puts "Triggering puppet runs on the following nodes:" puts puppet.discover.in_groups_of(3) do |nodes| puts " %-20s %-20s %-20s" % nodes end puppet.runonce printrpcstats else puts "ERROR: Could not determine a list of nodes to run" end |
The code between lines 14 and 46 just reads each line of the git post-receive hook STDIN and process them, you can read more about these hooks @ git-scm.com.
For each b path in the commit I parse its path based on puppet module conventions, node names, my hiera structure and some specific aspects of my file layouts. These end up in the @matched_modules, @matched_nodes and @matched_facts arrays.
MCollective 2.0.0 will let you supply node names not just from network based discovery but from any source really. Here I get node names from things like my node files, file names in iptables rules and such. Version 2.0.0 also supports a new query language for discovery which we use here. The goal is to do a network discovery only when I have non specific data like class names – if I found just a list of node names I do not need to do go out to the network to do discovery thanks to the new abilities of MCollective 2.0.0
In lines 48 to 90 I create a MCollective client to the puppetd agent, discover matching nodes and do the puppet runs.
If I found any code in the git push that matched either classes or facts I need to do a full MCollective discover based on those to get a node list. This is done using the new compound filtering language, the filter will look something like:
/some_class/ or some::other::class or fact=value |
But this expensive network wide discovery is only run when there are facts or classes matched out of the commit.
Line 72 will supply the combined MCollective discovered nodes and node names discovered out of the code paths as discovery data which later in line 85 will get used to trigger the runs.
The end result of this can be seen here, the commit matched only 5 out of my 25 machines and only those will be run:
$ git push origin master Counting objects: 13, done. Delta compression using up to 4 threads. Compressing objects: 100% (6/6), done. Writing objects: 100% (7/7), 577 bytes, done. Total 7 (delta 4), reused 0 (delta 0) remote: Commit on master remote: common/modules/mcollective/manifests/client.pp remote: remote: Files matched classes: mcollective::client remote: remote: Triggering puppet runs on the following nodes: remote: remote: node1 node2 node3 remote: node4 node5 remote: remote: 5 / 5 remote: remote: Finished processing 5 / 5 hosts in 522.15 ms To git@git:puppet.git 7590a60..10ee4da master -> master |
