HEP Workshop (2011)
This is the preparation wiki for the HEP track 2011. We collect topics, the developing agenda and some facts concerning the hands-on.
Requirements
For the hands-on we will have a few requirements, a laptop for login and credentials provided for the school should be enough.
The workshop will have two parts:
- Some presentations with discussions
Agenda
- Some hands-on for selected topics
Hands-on
We want to install LHC VO software on VM using CVMFS. Although for the examples not strictly need, we want to install a Squid server for caching in addition. No everyone needs his/her own. We might install one per two or four. Finally we try a very simple CMS example locally to verify our installation. We also test, if the CMS European redirector works, if our file vanishes locally.
Setup a (little) Squid server
Available hosts:
* gks-1-116 * gks-1-117
Access credentials will be provided. So we need two small teams to install them.
A Squid server introduces another cache layer at the local site. The installation for a small setup is very simple. Larger sites need some considerations (not covered here.)
We follow basically the deployment example outlined here: https://twiki.cern.ch/twiki/bin/view/PDBService/SquidRPMsTier1andTier2
Default configuration values of the RPMs can be overritten, but we do not need this. Therefore we just install the RPM:
rpm -ivh http://grid-deployment.web.cern.ch/grid-deployment/flavia/frontier-squid-2.7.STABLE9-3.7.sl5.x86_64.rpm
We do need to adjust some setting in /etc/squid/custumze.sh
awk --file `dirname $0`/customhelps.awk --source '{ setoption("acl NET_LOCAL src", "141.52.0.0/255.255.0.0") setoption("cache_mem", "256 MB") setoptionparameter("cache_dir", 3, "10000") print }'
Note the change in the NET_LOCAL line. This opens a certain netmask for allowed clients.
/sbin/service frontier-squid start
Do some testing with fnget.py:
./fnget.py --url=http://cmsfrontier.cern.ch:8000/Frontier/Frontier --sql="select 'test' from dual"
Look at the output. You should read the 'test' string at the end of the output. Change 'test' to another string e.g. your name. You should read it again at the output.
Now enable your squid like this:
export http_proxy=http://<Your-Squid-Host>:3128
Now repeat the test with fnget.py and have a look in the logs
cat /var/log/squid/access.log
In the first attempt it should look like this
200 843 TCP_MISS:DIRECT 38 "fnget.py 1.5"
For the second query it should look like this
200 842 TCP_MEM_HIT:NONE 0 "fnget.py 1.5"
Installing CernVMFS
This will be a challenge today, since the main expert for this part cannot be here today. We have to work along the documentation.
Just like in the real world....
Set CMS_PATH to a proper place
export CMS_PATH=/opt/cms mkdir -p $CMS_PATH
Xrootd fall back CMS example
CMS file names are organized in LFNs (Logical File Names). They are mapped to PFNs (Physical File Names) by means of a Trivial File Catalogue (TFC) on each site. The mapping can be programmed depending on the access protocol. This is a simple example:
$CMS_PATH/SITECONF/local/Phedex/storage.xml
<storage-mapping> <lfn-to-pfn protocol="direct" path-match="/+(.*)" result="/<YOUR_STORAGE_ROOT>/GridKaSchool2011/$1"/> <pfn-to-lfn protocol="direct" path-match="/<YOUR_STORAGE_ROOT>/GridKaSchool2011/(.*)" result="/$1"/> <lfn-to-pfn protocol="xrootd" path-match="/+(.*)" result="root://xrootd.ba.infn.it//$1"/> <pfn-to-lfn protocol="xrootd" path-match="/(.*)" result="/$1"/> </storage-mapping>
Which protocol is used can be configured in another configuration file:
$CMS_PATH/SITECONF/local/JobConfig/site-local-config.xml
<site-local-config> <site name="local"> <event-data> <catalog url="trivialcatalog_file:<YOUR CMS_ROOT_PATH>/SITECONF/local/PhEDEx/storage.xml?protocol=direct"/> <catalog url="trivialcatalog_file:<YOUR_CMS_ROOT_PATH>/SITECONF/local/PhEDEx/storage.xml?protocol=xrootd"/> </event-data> </site> </site-local-config>
Put the example file into you local directory. Create proper paths, you can later play around with the mappings.
cd <YOUR_STORAGE_ROOT> mkdir -p store/user/wissingc cd store/user/wissingc/ wget http://grid.desy.de/GKS2011/RelValTTbar_RECO_424.root cd <to your working area e.g. $HOME>
Prepare the little Demo Analyser (taken from the CMS workbook)
source <CMS_ROOT_PATH>/cmsset_default.sh [CMS_ROOT_PATH is /cvmfs/cms.cern.ch]
cmsrel CMSSW_4_2_5 cd CMSSW_4_2_5/src cmsenv
At this place you can test the LFN to PFN resolving. CMSSW provides a tool that reads your TFC and does the resolving
edmFileUtil -d <LFN>
Once the resolving works, we can start the real little exercise.
mkdir Demo cd Demo mkedanlzr DemoAnalyzer cd DemoAnalyzer scram b
The job configuration demoanalyzer_cfg.py should look like this:
import FWCore.ParameterSet.Config as cms process = cms.Process("Demo") process.load("FWCore.MessageService.MessageLogger_cfi") process.maxEvents = cms.untracked.PSet( input = cms.untracked.int32(-1) ) process.source = cms.Source("PoolSource", # replace 'myfile.root' with the source file you want to use fileNames = cms.untracked.vstring( '/store/user/wissingc/RelValTTbar_RECO_424.root' ) ) process.demo = cms.EDAnalyzer('DemoAnalyzer' ) process.p = cms.Path(process.demo)
Note that the /store/user/... entry is an LFN, which gets resolved by the TFC.
Run the CMS excutable
cmsRun demoanalyzer_cfg.py
It should run the demo code and open the file from the local path.
No we test the xrootd fallback. Therefore we rename the original file:
Go to the directory where it is located and rename it
mv RelValTTbar_RECO_424.root Hidden_RelValTTbar_RECO_424.root
Run the CMS demo again and have a look at the output! Since the file cannot be accessed any longer, it will be opened remotely via an xrootd redirector, which redirects the client to an SE, that actually has the file.
cmsRun demoanalyzer_cfg.py