Just a quick blog post for those who follow me here to get notified about new releases of MCollective. I just released version 0.4.2 which brings in big improvements for Debian packages, some tweaks to command line and a bug fix in SimpleRPC.
Read all about it at the Release Notes
Google Code does not provide it’s own usable export methods for projects so we need to make do on our own, there seems no sane way to back up the tickets but for SVN which includes the wiki you can use svnsync.
Here’s a little script to automate this, just give it a PREFIX of your choice and a list of projects in the PROJECTS variable, cron it and it will happily keep your repos in sync.
It outputs its actions to STDOUT so you should add some redirect or redirect it from cron.
#!/bin/bash PROJECTS=( your project list ) PREFIX="/path/to/backups" [ -d ${PREFIX} ] || mkdir -p ${PREFIX} cd ${PREFIX} for prj in ${PROJECTS[@]} do if [ ! -d ${PREFIX}/${prj}/conf ]; then svnadmin create ${prj} ln -s /bin/true ${PREFIX}/${prj}/hooks/pre-revprop-change svnsync init file:///${PREFIX}/${prj} http://${prj}.googlecode.com/svn fi svnsync sync file:///${PREFIX}/${prj} done |
I stongly suggest you backup even your cloud hosted data.
I have a QNAP TS-209 Nas device. It’s a Linux based appliance with 2 hot swap drives.
It has now died by the looks of it, QNAP support has been utterly useless to say the least but I have pretty much resolved to just replacing this unit even if they are able to resurrect it. The problem with the 1xx and 2xx range of QNAP is that its some weird CPU architecture and to enable huge files on them they had to patch the ext3 file system.
The end result is that while the devices are advertised as being ext3 they are in-fact a patched ext3 and you cannot just mount them in a Linux machine. They have also now stopped selling this series of machine so should yours ever die you are just plainly out of luck. QNAP have made a live cd available that’s similarly patched so you should have some hope if you are really in trouble.
In my case the device seem to have also totally corrupted the drives when it died so even in the Live CD scenario both are dead. It seems the SATA interface has gone rather than the disks, the moment I put a disk in it seems the CPU is totally kept busy dealing with blocking I/O requests, out of a 1000 pings about 20 will get replies – and those will be 30 second response times.
This brings me to several points:
I’ve now spent the last 2 or so days duplicating my backups so I am redundantly covered while I look for a replacement. I’d have liked to not buy another QNAP but it’s really unfortunate that they do seem to have the best range of products in this space. All the vendors seem to have stopped selling 2 drive units so I am down to getting a 4 drive QNAP TS-439 now, this will set me back almost GBP900 but will give me 2TB of mirrored space, apple Time Machine backups for all my macs etc, pricey but important given that this is all my photos and music.
In the same week it seems my Apple iMac 24 inch has had similar problems. It isn’t booting, it seems a similar problem has afflicted it, I am not getting the usual I/O errors I saw on other macs when their drives died instead it’s other I/O timeouts that suggest more it’s the controller and not the drives. Thankfully I have Apple Care so it’s in for a free fixup. I do not have backups of this machine – that’s by design – since I keep all my data on servers on the internet and those are backed up off-site nightly. My desktops tend to be disposable and simply terminals to online data even browser bookmarks are stored online. The only thing lost on this machine would be chat logs and browser history, nothing else. I need to make some kind of plan with chat logs as those do tend to be more and more important these days.
So to sum up, even if you have multi redundancy in your drives in a NAS you must still do your backups, it’s easy with QNAPs to even do it off-site as you can rsync to a remote location or even sync to Amazon S3. Of course they also have USB ports so you can place files on an external drive.
A few days ago I released Marionette Collective version 0.4.0 and today I released 0.4.1. This release branch introduce a major new feature called Simple RPC.
In prior releases it took quite a bit of ruby knowledge to write a agent and client. In addition clients all ended up implementing their own little protocols for data exchange. We’ve simplified agents and clients and we’ve created a standard protocol between clients and agents.
Standard protocols between clients and agents means we have a standard one-size-fits-all client program called mc-rpc and it opens the door to writing simple web interfaces that can talk to all compliant agents. We’ve made a test REST Simple RPC bridge as an example.
Writing a client can now be done without all the earlier setup, command line parsing and so forth, it can now be as simple as:
require 'mcollective' include MCollective::RPC mc = rpcclient("rpctest") printrpc mc.echo(:msg => "Welcome to MCollective Simple RPC") printrpcstats |
This simple client has full discovery, full –help output, and takes care of printing results and stats in a uniform way.
This should make it much easier to write more complex agents, like deployers that interact with packages, firewalls and services all in a single simple script.
We’ve taken a better approach in presenting the output from clients now, instead of listing 1000 OKs on success it will now only print whats failing.
Output from above client would look something along these lines:
$ hello.rb * [ ============================================================> ] 43 / 43 Finished processing 43 / 43 hosts in 392.60 ms |
As you can see we have a nice progress indicator that will work for 1 or 1000 nodes, you can still see status of every reply by just running the client in verbose – which will also add more detailed stats at the end.
Agents are also much easier, here’s a echo agent:
class Rpctest<RPC::Agent def echo_action validate :msg, :shellsafe reply.data = request[:msg] end end |
You can get full information on this new feature here. We’ve also created a lot of new wiki docs about ActiveMQ setup for use with MCollective and we’ve recorded a new introduction video here.
MCollective is a framework for writing RPC style tools that talk to a cloud of servers, till now doing that has been surprisingly hard for non ruby coders. The reason for this is that I was focussing on getting the framework built and feeling my way around the use cases.
I’ve now spent 2 days working on simplifying actually writing agents and consumers. This code is not released yet – just in SVN trunk – but here’s a taster.
First writing an agent should be simple, here’s a simple ‘echo’ server that takes a message as input and returns it back.
class Rpctest<RPC::Agent # Basic echo server def echo_action(request, reply) raise MissingRPCData, "please supply a :msg" unless request.include?(:msg) reply.data = request[:msg] end end |
This creates an echo action, does a quick check that a message was received and sends it back. I want to create a few more validators so you can check easily if the data passed to you is sane and secure if you’re doing anything like system() calls with it.
Here’s the client code that calls the echo server 4 times:
#!/usr/bin/ruby require 'mcollective' include MCollective::RPC rpctest = rpcclient("rpctest") puts "Normal echo output, non verbose, shouldn't produce any output:" printrpc rpctest.echo(:msg => "hello world") puts "Flattened echo output, think combined 'mailq' usecase:" printrpc rpctest.echo(:msg => "hello world"), :flatten => true puts "Forced verbose output, if you always want to see every result" printrpc rpctest.echo(:msg => "hello world"), :verbose => true puts "Did not specify needed input:" printrpc rpctest.echo |
This client supports full discovery and all the usual stuff, has pretty –help output and everything else you’d expect in the clients I’ve supplied with the core mcollective. It caches discovery results so above code will do one discovery only and reuse it for the other calls to the collective.
When running you’ll see a twirling status indicator, something like:
- [5 / 10] |
This will give you a nice non scrolling indicator of progress and should work well for 100s of machines without spamming you with noise, at the end of the run you’ll get the output.
The printrpc helper function tries its best to print output for you in a way that makes sense on large amounts of machines.
The script above gives the following output when run in non-verbose mode:
$ rpctest.rb --with-class /devel/
Normal echo output, non verbose, shouldn't produce any output:
Forced verbose output, if you always want to see every result:
dev1.your.com : OK
"hello world"
dev2.your.com : OK
"hello world"
dev3.your.com : OK
"hello world"
Flattened echo output, think combined 'mailq' usecase:
hello world
hello world
hello world
Did not specify needed input:
dev1.your.com : please supply a :msg
dev2.your.com : please supply a :msg
dev3.your.com : please supply a :msg |
Still some work to do, specifically stats needs a rethink in a scenario where you are making many calls such as in this script.
This will be in mcollective version 0.4 hopefully out early January 2010
