{"id":1763,"date":"2010-09-18T16:35:08","date_gmt":"2010-09-18T15:35:08","guid":{"rendered":"http:\/\/www.devco.net\/?p=1763"},"modified":"2010-09-19T08:45:29","modified_gmt":"2010-09-19T07:45:29","slug":"puppet_search_engine_with_mcollective","status":"publish","type":"post","link":"https:\/\/www.devco.net\/archives\/2010\/09\/18\/puppet_search_engine_with_mcollective.php","title":{"rendered":"Puppet Search Engine with MCollective"},"content":{"rendered":"

The current supported way of making a Puppet node aware of the state of other nodes is Exported Resources<\/a>, there’s some movement in this space see for example a recent thread on the developers list<\/a>.<\/p>\n

I’ve personally never found exported resources to be a good fit for me, I have many masters spread over long network distances and they often operate in a split brain scenario where the masters couldn’t see each other. This makes using something like MySQL as a database hard since you’d effectively need a single write point and replication is point to point.<\/p>\n

There are other problems too, consider a use case where you have modules for phpbb, wordpress, rails apps, phpmyadmin and several others that need to talk to the master database. Maybe you also want to show in the motd a resource list for customers. <\/p>\n

Every one of these modules have configuration parameters that differ in some way or the other. The exported resources model kind of expect you to export from the MySQL Master the configs for all the apps using the database, this isn’t viable when you don’t know what all will talk to this MySQL server and the resource abstraction doesn’t really work well when the resulting config might need to look like this:<\/p>\n

<\/p>\n

\r\n$write_database = \"db1.your.net\";\r\n$read_databases = \"db2.your.net\", \"db3.your.net\", \"db4.your.net\";\r\n<\/pre>\n
<\/code><\/p>\n
Filling in that array with exported resources is pretty hard.<\/p>\n
So I favor searching nodes and walking the results.  I tend to use extlookup<\/a> for this and just maintain the hosts lists by hand, this works but it’s maintained by hand, there has to be a better way.<\/p>\n
I wrote a mcollective registration system<\/a> that puts all node data in MongoDB<\/a> instances.  <\/p>\n
Simply using mcollective to manage the data store has several advantages:<\/p>\n