One of the goals of building the SimpleRPC framework and the overall speed of MCollective is to create interactive tools to manage your infrastructure in a way that it all just seems like a single point of entry with one machine. I’ve blogged a bit about this before with how I manage Exim clusters.
I’ve recently built a deployer for a client that does some very specific things with their FastCGI, packages and monitoring in a way that is safe for developers to use. I’ve made a sanitized demo of it that you can see below. It’s sanitized in that the hostnames are replaced with hashes and some monitoring details removed but you’ll get the idea.
With the new SimpleRPC system in MCollective we have a simple interface to creating agents. The way to call an agent would be:
$ mc-rpc service status service=httpd
$ mc-rpc service status service=httpd
This is all fine and well and easy enough, however it requires you to know a lot. You need to know there’s a status action and you need to know it expects a service argument, not great.
I’m busy adding the ability for an agent to register its metadata and interface so that 3rd party tools can dynamically generate useful interfaces.
A sample registration for service agent is:
register_meta(:name=>"SimpleRPC Service Agent",
:description=>"Agent to manage services using the Puppet service provider",
:author=>"R.I.Pienaar",
:license=>"GPLv2",
:version=>1.1,
:url=>"http://mcollective-plugins.googlecode.com/",
:timeout=>60)["start", "stop", "restart", "status"].eachdo|action|
register_input(:action=> action,
:name=>"service",
:prompt=>"Service Name",
:description=>"The service to #{action}",
:type=> :string,
:validation=>'^[a-zA-Z\-_\d]+$',
:maxlength=>30):
register_meta(:name => "SimpleRPC Service Agent",
:description => "Agent to manage services using the Puppet service provider",
:author => "R.I.Pienaar",
:license => "GPLv2",
:version => 1.1,
:url => "http://mcollective-plugins.googlecode.com/",
:timeout => 60)
["start", "stop", "restart", "status"].each do |action|
register_input(:action => action,
:name => "service",
:prompt => "Service Name",
:description => "The service to #{action}",
:type => :string,
:validation => '^[a-zA-Z\-_\d]+$',
:maxlength => 30):
This includes all the meta data, versions, timeouts, validation of inputs, prompts and help text for every input argument.
I just released version 0.4.3 of mcollective which brings a new auditing capability to SimpleRPC. Using the auditing system you can log to a file on each host every request or build a centralized auditing system for all requests on all nodes.
We ship a simple plugin that logs to the local harddrive but there is also a community plugin that creates a centralized logging system running over MCollective as a transport.
This is the kind of log the centralized logger will produce:
Here we see 3 nodes that got a request to add 114.255.136.120 to their local firewall. The request was sent by UID 500 on the machine ids1.my.net. The request is of course the same everywhere so the request id is the same on every node, the log shows agent and all parameters passed.
Just a quick blog post for those who follow me here to get notified about new releases of MCollective. I just released version 0.4.2 which brings in big improvements for Debian packages, some tweaks to command line and a bug fix in SimpleRPC.
A few days ago I released Marionette Collective version 0.4.0 and today I released 0.4.1. This release branch introduce a major new feature called Simple RPC.
In prior releases it took quite a bit of ruby knowledge to write a agent and client. In addition clients all ended up implementing their own little protocols for data exchange. We’ve simplified agents and clients and we’ve created a standard protocol between clients and agents.
Standard protocols between clients and agents means we have a standard one-size-fits-all client program called mc-rpc and it opens the door to writing simple web interfaces that can talk to all compliant agents. We’ve made a test REST Simple RPC bridge as an example.
Writing a client can now be done without all the earlier setup, command line parsing and so forth, it can now be as simple as:
require'mcollective'includeMCollective::RPC
mc = rpcclient("rpctest")
printrpc mc.echo(:msg=>"Welcome to MCollective Simple RPC")
printrpcstats
require 'mcollective'
include MCollective::RPC
mc = rpcclient("rpctest")
printrpc mc.echo(:msg => "Welcome to MCollective Simple RPC")
printrpcstats
This simple client has full discovery, full –help output, and takes care of printing results and stats in a uniform way.
This should make it much easier to write more complex agents, like deployers that interact with packages, firewalls and services all in a single simple script.
We’ve taken a better approach in presenting the output from clients now, instead of listing 1000 OKs on success it will now only print whats failing.
Output from above client would look something along these lines:
$ hello.rb
* [ ============================================================> ] 43 / 43
Finished processing 43 / 43 hosts in 392.60 ms
$ hello.rb
* [ ============================================================> ] 43 / 43
Finished processing 43 / 43 hosts in 392.60 ms
As you can see we have a nice progress indicator that will work for 1 or 1000 nodes, you can still see status of every reply by just running the client in verbose – which will also add more detailed stats at the end.
Agents are also much easier, here’s a echo agent:
class Rpctest<RPC::Agentdef echo_action
validate :msg, :shellsafe
reply.data = request[:msg]endend
class Rpctest<RPC::Agent
def echo_action
validate :msg, :shellsafe
reply.data = request[:msg]
end
end
You can get full information on this new feature here. We’ve also created a lot of new wiki docs about ActiveMQ setup for use with MCollective and we’ve recorded a new introduction video here.
