Middleware for Systems Administration

I spoke a bit on the puppet IRC channel about my middleware based systems administration tool, I made a video to demo it below.

The concept is that I use publish / subscribe middleware – ActiveMQ with Stomp in my case – to do one-off administration. Unlike using Capistrano or some of those tools I do not need lists of machines or visit each machine with a request because the network supports discovery and a single message to the middleware results in 10s or 100s or 1000s of machines getting the message.

This means any tasks I ask to be done happens in parallel on any number of machines typically I see ~100 machines finish the task in the same time as 1 machine would and no need for SSH or anything like that.

The app server and client libs I wrote take away all the complexities of the middleware and takes care of crypto signing requests, only responding to requests that has been signed properly etc, serializing and deserialization of data etc.

Discovery is built in and it supports puppet classes and facts and a few other criteria I use for my own systems so there is no need to build any kind of system that keeps track of what machines I have with what version of operating system etc. As long as is on the middleware I can find it.

The bulk – timeout handling and so forth removed – of the ping app you see in the demo can be seen here, client:

client = Stomphost::Client.new(config)
client.sendreq(Time.now.to_f, "echo")
 
loop do
    resp = client.receive 
    elapsed = (Time.now.to_f - resp[:body]) * 1000
end

And the agent is just this:

module Stomphost
    module Agent
        class Echo
            def handlemsg(msg)
                msg[:body]
            end
        end
    end
end

You can see that even data types like the float will flow cleanly through end to end.

Watch the video, I mention my uses cases but it includes distributed Exim administration, package updates, services restarts, iptables management and much more.

UPDATE: This code has now been released as an Open Source Apache 2 licenced project at marionette-collective.org