Testing CREAM CE
Torque
the command qstat -q should give an output similar to
qstat -q server: gks-0-063.fzk.de Queue Memory CPU Time Walltime Node Run Que Lm State ---------------- ------ -------- -------- ---- --- --- -- ----- dech -- 48:00:00 72:00:00 -- 0 0 -- E R ----- ----- 0 3
The line containing server: should show the hostname of your Torque batchsystem server.
pbsnodes -a shows all information on available worker nodes. Run this command and have a look at the output which should be similar to the (manually formatted) one below. gks-1-123.fzk.de
state = free np = 1 properties = lcgpro ntype = cluster status = opsys=linux, uname=Linux gks-1-123 2.6.18-194.8.1.el5 #1 SMP Thu Jul 1 16:05:53 EDT 2010 x86_64, sessions=? 0, nsessions=? 0, nusers=0, idletime=176199, totmem=11868608kb, availmem=11735552kb, physmem=4056784kb, ncpus=4, loadave=0.00, netload=393808345, state=free, jobs=, varattr=, rectime=1283337303
Try to submit a job as a user, e.g.
su - dech004 cat > test.sh <<+ #!/bin/bash echo "I am \`whoami\` on host \`hostname\`." sleep 15 exit 0 + qsub -q test test.sh
This job should show up under qstat and the output must appear in the home directory
test.sh.e<job_id> test.sh.o<job_id>
If the output is not available after the job has finished check the connection from WN->CE. The user has to be able to use a password free ssh and scp from the WN to the CE. You can try this with an interactive job or directly on the Worker node (as a user).
su - dech004 qsub -I -q test ssh <CE host>
Information System
The information system can be tested via a ldap command
ldapsearch -xLLL -b mds-vo-name=resource,o=grid -p 2170 -h <CE hostname>
If you get no output check that the ldap service is running correctly on the CREAM compute element.
/etc/init.d/bdii status
Except for the static information all dynamic values have to be published correctly. A certain clue for a broken information plugin is the number 444444 for GlueCEStateWaitingJobs. To check all dynamic information you can have a look at the files in directory /opt/glite/var/cache/gip/*. For every plugin and provider one file with the dynamic information is created. Check that every file contains the right information and is not empty.
Job Submission
For the job submission please log on an User Interface, e.g. gks-1-135.fzk.de (use the account gks??? given to you in your school bag). You need a valid voms proxy for this test.
voms-proxy-init --voms dech
NOTE: All input and output files must be copied from/to a gridftp server.
Make your own directory on a gridftp server
export WHOAMI=`whoami` uberftp f01-014-110-e.gridka.de "mkdir /pnfs/gridka.de/dech/disk-only/$WHOAMI"
As an example jdl for dech you can use
cat > test.jdl <<+ Executable = "/bin/bash"; Arguments = "test.sh"; StdOutput = "std.out"; StdError = "std.err"; InputSandbox="gsiftp://f01-014-110-e.gridka.de/pnfs/gridka.de/dech/disk-only/$WHOAMI/test.sh"; OutputSandbox = {"std.out", "std.err"}; OutputSandboxBaseDestURI="gsiftp://f01-014-110-e.gridka.de/pnfs/gridka.de/dech/disk-only/$WHOAMI/"; +
The test.sh script on the storage contains
#!/bin/bash echo 'Hello World!' echo "hostname=`hostname`" uname -a whoami exit 0
The job submission can be tested in two ways. Either do a job submission via (ICE enables) WMS or you can use a CREAM CLI available on the UIs/login hosts.
Usage of the CREAM CLI:
glite-ce-job-submit -a -r <ce host>:8443/cream-pbs-<queue> -o IDs test.jdl glite-ce-job-status -i IDs
After the job has finished do a
glite-ce-job-purge -i IDs
The output files can listed with
uberftp f01-014-110-e.gridka.de "ls /pnfs/gridka.de/dech/disk-only/$WHOAMI"
copied to the UI and deleted on the server with
globus-url-copy gsiftp://f01-014-110-e.gridka.de/pnfs/gridka.de/dech/disk-only/$WHOAMI/std.out file:///home/$WHOAMI/std.out globus-url-copy gsiftp://f01-014-110-e.gridka.de/pnfs/gridka.de/dech/disk-only/$WHOAMI/std.err file:///home/$WHOAMI/std.err uberftp f01-014-110-e.gridka.de "rm /pnfs/gridka.de/dech/disk-only/$WHOAMI/std.out" uberftp f01-014-110-e.gridka.de "rm /pnfs/gridka.de/dech/disk-only/$WHOAMI/std.err"
Now have a look at Administration of the CREAM CE
Go back to gLite Administration Course, CREAM CE