I am quite a big fan of CentOS, I run all my own systems on it and its my first recommendation for clients.
I’ve been using Red Hat since their Halloween release 31 October 1994. The major thing you notice about it is that it stays consistent and sticks to standards – usually existing ones without trying to make its own.
Consistency is really important in my mind from an Operating System, you do not want to redesign your own systems just because you go up a point release or even a major release. Red Hat more than certain other popular Linux Distribution has been providing this for years now.
CentOS gets the benefit of all of this plus provides you with a reasonably easy upgrade path to Red Hat should your investors for example require this, it maintains binary compatibility with Red Hat proper so you can rest assured that should the need arise to upgrade to a supported distribution you at least won’t need to redevelop your own code.
Why anyone would pay Red Hat support is another matter, I cannot remember a single instance of being satisfied with their support in the various places I’ve used them.
Anyway, 62976 processor core CentOS machines? Sun just announced their Constellation System HPC cluster based on their new blades as a backbone. From a searchdatacenter.com article the following:
The Sun Constellation CentOS Linux Cluster, named Ranger, will have 3,936 nodes, 123 terabytes of memory and 62,976 processor cores from AMD Opteron quad-core processors. The system is specifically designed to support very large science and engineering computing, according to TACC.