If you have a bproc cluster, and want to test xcpu, it's very easy. In fact, at LANL, we made bproc cluster that also, always, started xcpufs up as well. We thus had a combined bproc/xcpu cluster.

Note that there is not need to change your kernel for xcpu! Older xcpu systems required kernels with v9fs; this requirement is gone.

First, get the xcpu-auth branch. This provides an xcpu with strong authentication.

svn co svn://xcpu.org/sxcpu/branches/xcpu-auth

cd xcpu-auth

make

make install

At this point you have xcpu-auth on your machine.

Now xcpufs does have authentication and so needs keys. The system uses public/private keys, generated by ssh-keygen. xcpufs, which runs on the xcpu servers, a.k.a. compute nodes, needs the public key. The various tools need the private key.

So, you're going to have to create the keys. Also, for purposes of convenience, we install the key for root as the node comes up. So both public and private keys are installed in the initrd; the private key on the compute nodes is used once (as part of boot) and then deleted. This removal makes it difficult for bad users to try to steal the private key.

So, make the key:

ssh-keygen -t rsa -f admin_key

Now, mkdir /etc/xcpu, and (at minimum) cp the admin_key to /etc/xcpu on the bproc master.

You need the key on your slave nodes, so:
bpsh -a mkdir /etc/xcpu
bpsh -a dd of=/etc/xcpu/admin_key.pub < admin_key.pub

That puts the key out.

Now:
bpsh -a xcpufs &

Unlike bproc, xcpu is very secure. It knows nothing about users and groups until you tell it. It won't believe you unless you have the admin key to authenticate yourself to it.

So, you need to tell xcpufs daemons about yourself. We don't (yet) have a -a equivalent on xcpu yet (we had it, but broke it, and are fixing it) so for now here is the easiest way to do this. If it seems weird, it is.

First, for each bproc node, make an entry in /etc/hosts of the form
IP-address n
e.g. 10.0.0.1 n1

Note that it doesn't have to be n1, it can be p1 or somelongnodename1, just make sure it ends in a number, and it's a lot easier if the number matches the bproc slave number.

Next, do this (assuming your highest numbered node is, e.g., 64):
xgroupset 'n[1-64]' root 0
xuserset 'n[1-64]' root 0 root ~root/.ssh/id_rsa.pub At this point you can just xrx to it directly (as root):
xrx n1 date
or
xrx 'n[1-64]' date

To add more groups, since we only have the root group, you need to run the xgroupset command, in this case the name somegroup with gid 567:
xgroupset node somegroup 567

To add more users, e.g. add rminnich with uid 525 and group somegroup, you need to add their information and transfer the public key to xcpufs:
xuserset node rminnich 525 somegroup ~rminnich/.ssh/id_ras.pub

Note that we added 'somegroup' for rminnich in that earlier xgroupset command.