MCollective Security with ActiveMQ

by R.I. Pienaar | Nov 18, 2009 | Code

As part of rolling out mcollective you need to think about security. The various examples in the quick start guide and on this blog has allowed all agents to talk to all nodes all agents. The problem with this approach is that should you have untrusted users on a node they can install the client applications and read the username/password from the server config file and thus control your entire architecture.

Since revision 71 of trunk the structure of messages has changed to be compatible with ActiveMQ authorization structure, I’ve also made the structure of the message targets configurable. The new default format is compatible with ActiveMQ wildcard patterns and so we can now do fine grained controls over who can speak to what.

General information about ActiveMQ Security can be found on their wiki.

The default message targets looks like this:

/topic/mcollective.agentname.command
/topic/mcollective.agentname.reply

The nodes only need read access to the command topics and only need write access to the reply topics. The examples below also give them admin access so these topics can be created dynamically. For simplicity we’ll wildcard the agent names, you could go further and limit certain nodes to only run certain agents. Adding these controls effectively means anyone who gets onto your node will not be able to write to the command topics and so will not be able to send commands to the rest of the collective.

There’s one special case and that’s the registration topic, if you want to enable the registration feature you should give the nodes access to write on the command channel for the registration agent. Nothing should reply on the registration topic so you can limit that in the ActiveMQ config.

We’ll let mcollective log in as the mcollective user, create a group called systemusers, we’ll then give the systemsuser group access to run as a typical registration enabled mcollective node.

The rip user is a mcollective admin and can create commands and receive replies.

First we’ll create users and the groups.

<simpleAuthenticationPlugin>
 <users>
  <authenticationUser username="mcollective" password="pI1SkjRi" groups="mcollectiveusers,everyone"/>
  <authenticationUser username="rip" password="foobarbaz" groups="admins,everyone"/>
 </users>
</simpleAuthenticationPlugin>

Now we’ll create the access rights:

<authorizationPlugin>
  <map>
    <authorizationMap>
      <authorizationEntries>
        <authorizationEntry queue="mcollective.>" write="admins" read="admins" admin="admins" />
        <authorizationEntry topic="mcollective.>" write="admins" read="admins" admin="admins" />
        <authorizationEntry topic="mcollective.*.reply" write="mcollectiveusers" admin="mcollectiveusers" />
        <authorizationEntry topic="mcollective.registration.command" write="mcollectiveusers" read="mcollectiveusers" admin="mcollectiveusers" />
        <authorizationEntry topic="mcollective.*.command" read="mcollectiveusers" admin="mcollectiveusers" />
        <authorizationEntry topic="ActiveMQ.Advisory.>" read="everyone,all" write="everyone,all" admin="everyone,all"/>
      </authorizationEntries>
    </authorizationMap>
  </map>
</authorizationPlugin>

You could give just the specific node that runs the registration agent access to mcollective.registration.command to ensure the secrecy of your node registration.

Finally the nodes need to be configured, the server.cfg should have the following at least:

topicprefix = /topic/mcollective
topicsep = .
plugin.stomp.user = mcollective
plugin.stomp.password = pI1SkjRi
plugin.psk = aBieveenshedeineeceezaeheer

For my clients I can use the ability to configure the user details in my shell environment:

export STOMP_USER=rip
export STOMP_PASSWORD=foobarbaz
export STOMP_SERVER=stomp1
export MCOLLECTIVE_PSK=aBieveenshedeineeceezaeheer

And finally the rip user when logged into a shell with these variables have full access to the various commands. You can now give different users access to the entire collective or go further and give a certain admin user access to only run certain agents by limiting the command topics they have access to. Doing the user and password settings in shell environments means it’s not kept in any config file in /etc/ for example.

ActiveMQ Clustering

by R.I. Pienaar | Nov 10, 2009 | Code

As part of deploying MCollective + ActiveMQ instead of my old Spread based system I need to figure out a multi location setup, the documentation says I’d possible so I thought I better get down and figure it out.

In my case I will have per-country ActiveMQ’s, I’ve had the same with Spread in the past and it’s proven reliable enough for my needs, each ActiveMQ will carry 30 or so nodes.

ActiveMQ Cluster

The above image shows a possible setup, you can go much more complex, you can do typical hub-and-spoke setups, a fully meshed setup or maybe have a local one in your NOC etc, ActiveMQ is clever enough not to create message loops or storms if you create loops so you can build lots of resilient routes.

ActiveMQ calls this a Network of Brokers and the minimal docs can be found here. They also have docs on using SSL for connections, you can encrypt the inter DC traffic using that.

I’ll show sample config below of the one ActiveMQ node, the other would be identical except for the IP of it’s partner.  The sample uses authentication between links as I think you really should be using auth everywhere.

   <broker xmlns="http://activemq.org/config/1.0" brokerName="your-host" useJmx="true"
      dataDirectory="${activemq.base}/data">
 
      <transportConnectors>
         <transportConnector name="openwire" uri="tcp://0.0.0.0:6166"/>
         <transportConnector name="stomp"   uri="stomp://0.0.0.0:6163"/>
      </transportConnectors>

These are basically your listeners, we want to accept Stomp and OpenWire connections.

Now comes the connection to the other ActiveMQ server:

<networkConnectors>
   <networkConnector name="amq1-amq2" uri="static:(tcp://192.168.1.10:6166)" userName="amq" password="Afuphohxoh"/>
</networkConnectors>

This sets up a connection to the remote server at 192.168.1.10 using username amq and password Afuphohxoh. You can also designate failover and backup links, see the docs for samples. If you’re building lots of servers talking to each other you should give every link on every server a unique name. Here I called it amq1_amq2 for comms from a server called amq1 to amq2, this is a simple naming scheme that ensures things are unique.

Next up comes the Authentication and Authorization bits, this sets up the amq user and an mcollective user that can use the topic /topic/mcollective.*. More about ActiveMQ’s security model can be found here.

    <plugins>
      <simpleAuthenticationPlugin>
        <users>
          <authenticationUser username="amq" password="Afuphohxoh" groups="admins,everyone"/>
          <authenticationUser username="mcollective" password="pI1jweRV" groups="mcollectiveusers,everyone"/>
        </users>
      </simpleAuthenticationPlugin>
      <authorizationPlugin>
        <map>
          <authorizationMap>
            <authorizationEntries>
              <authorizationEntry queue=">" write="admins" read="admins" admin="admins" />
              <authorizationEntry topic=">" write="admins" read="admins" admin="admins" />
              <authorizationEntry topic="mcollective.>" write="mcollectiveusers" read="mcollectiveusers" admin="mcollectiveusers" />
              <authorizationEntry topic="ActiveMQ.Advisory.>" read="everyone,all" write="everyone,all" admin="everyone,all"/>
            </authorizationEntries>
          </authorizationMap>
        </map>
      </authorizationPlugin>
    </plugins>
  </broker>

If you setup the other node with a setup connecting back to this one you will have bi-directional messages working correctly.

You can now connect your MCollective clients to either one of the servers and everything will work as if you had only one server. ActiveMQ servers will attempt reconnects regularly if the connection breaks.

You can also test using my generic stomp client that I posted in the past