Right now , there are three available iguana,frog and fireworks.Iguana and Fireworks are officially supported by CMS management and have behind small teams of developers. Frog has no CMS official support and is essentially the result of the work of a single person: Loic Quertenmont.
Why so many tools?
Considering that an ideal event display should allow the casual user:
Ok, I just need to see what a RECO dataset from Castor contains. What is the easiest way to do it?
(Complete and up-to-date instructions available from https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookFireworks )
First I suppose that you have a PC with adequate hardware resources. If not you are out of luck. Then you are on a network with a fast connection to Cern. In this case the easiest way (but not the fastest or the best concerning the quality of images) is:
ssh -l your_user_name lxplus /afs/cern.ch/cms/fireworks/cmsShow21/cmsShow -c /afs/cern.ch/cms/fireworks/cosmics.fwc rootfile (add rfio:/castor/cern.ch/cms at the beginning if the dataset is on castor)
This was easy to use but I want to have access to all data collections of a single event.What you suggest:
Always with the same caveat(adequate hardware+fast connection).
ssh -l your_user_name lxplus cd /tmp/your_user_name (or some other directory where you have lot of space) scramv1 p CMSSW CMSSW_2_1_10 (release compatible with the data you want to see) cd CMSSW_2_1_10/src project CMSSW eval `scramv1 runtime -csh` iguana rootfile (add rfio:/castor/cern.ch/cms at the beginning if dataset on castor) (select next event)You get an empty 3D window on the right and a huge tree on the left that describes everything you have in the event. The idea is that you check an entry and you get it displayed on the right window. You select the text of the entry and you get it printed in the small white window on the right bottom. This is true but unfortunately it is very easy to get stuck when you do this. So to get something done you should proceed with big caution and know in advance what you are doing. If you try to follow the instructions given below you should produce no disaster:(the first time it is better to start with an event that you know is interesting when you examined the sample with fireworks)
in the "Event" menu goto the desired event.You must write run number and event number correctly. select the tracks collections that interest you select "Reco Tracks Det" select "Draw style" "still draw style" "wireframe" with right mouse button on 3D window. select sistrip clusters and pixel clusters select hcal endcap position the imageNow you can start scanning events using the key CTRL+N
It is possible to create rz and rphi windows and to look at the event with multiple windows but unfortunately this can be done "safely" only with a local installation of iguana.And this installation requiring full CMSSW isn't straightforward.
Instead of installing locally iguana there are other ways to scan fast 3D events?
An alternative is to use frog. Frog consists of two components :a frog visualizer and a producer. The visualizer can be quickly installed locally. It displays events only
in a special format contained in .vis file created by the producer. Howevere there are some .vis files already available. . To see them:
cd frog_directory edit config.txt (change the URL in the line "InputVisFile=") ./frog
Frog seems promising.But how do I install it on my computer ?
(Complete and up-to-date instructions available from http://projects.hepforge.org/frog/ )
Under windows: download zip file. Uncompress it in some directory and you are in businnes! Alternatively download from CVS repository soft directory with all subdirectories. The file to execute is frog.exe.
Under linux: it can be more difficult. Although frog doesn't need CMSSW, installing it on a arbitrary linux distribution may sometimes require the installation of additional software. This consists of standard Linux libraries and,if this happens , you should know how to install these.
If instead, you are running linux slc4 on your computer and you can access CMSSW software then , by following the steps indicated below, you are sure to have the necessary libraries installed.
cd directory_with_enough_space project CMSSW scramv1 project CMSSW CMSSW_2_1_11 (choose always the last) cd CMSSW_2_1_11/src cvs co -r Version_1_104 -d Visualisation/Frog UserCode/FastOpenGlDisplayer (check on the frog site which is the last) cd Visualisation/Frog/soft/ make
What if I want to see what a RAW dataset contain?
In this case fireworks isn't of much help and iguana is too slow. The problem here is that you need to redo the reconstrunction using CMSSW. This being the case, then the fastest way to scan raw data is to use frog. You have to run a CMSSW cfg like the one needed by iguana but this will produce in the python directory a temporary .vis (InLive.vis) file that you can immediately inspect from a remote computer with the frog visualizer installed . You run the CMSSW job on lxplus and write this temporary file on $HOME/scratchx/ where it is accessible from every computer that can access locally /afs/cern.ch/user/. Here the recipe to follow:
(on lxplus , if not yet installed, you have to install the latest frog:) cd directory_with_enough_space project CMSSW scramv1 project CMSSW CMSSW_2_1_9 (choose always the last) cd CMSSW_2_1_9/src cvs co -r Version_1_104 -d Visualisation/Frog UserCode/FastOpenGlDisplayer (check on the frog site which is the last) scramv1 b cd Visualization/Frog/python (end of installation: to be done only once) cd frog_directory cd CMSSW_2_1_9/src project CMSSW eval `scramv1 runtime -csh` cd Visualization/Frog/python edit Frog_Analyzer_Cosmics_FromRaw.py indicating the name of input dataset cmsRun Frog_Analyzer_Cosmics_FromRaw.py (you should see the job printing a line for each event processed) (on your pc) (if not yet done) change InputViSFile to point to temporary file start frogYou can start looking at events as soon as they are processed on lxplus.Just push the R (for reload) key when you want the latest events to be accessible. Or even better, edit frog configuration file (config.txt) with these 3 parameters. These make frog always display the latest event in the .vis file and will reload .vis files every 1sec. (So displayed event will always be the latest available event in the .vis)
InputVisFile = {../python/InLive.vis}; // In order to load events in the tempory .vis file (Name InLive.vis is hardcoded, so you have to use this)
Event_Number = -1; // This will always display the latest event in the file
Event_Time = 1; // This will check the .vis file every 1 sec.
Can I use frog in the same way to look at RECO data?
Yes. The procedure is the same : replace only the file Frog_Analyzer_Cosmics_FromRaw.py with Frog_Analyzer_Cosmics.py
I have the latest release of Iguana installed locally on my pc. So it should be fast enough. How can I use it to look at RAW data?
(Complete and up-to-date instructions available from Tracker visualization wiki )
cd CMSSW_2_1_10/src/ eval `scramv1 runtime -csh` cmscvsroot CMSSW cvs login (give 98passwd as password) cvs co VisReco/VisCustomTracker/test cd VisReco/VisCustomTracker/test (edit iguanaraw.cfg changing input dataset) iguana iguanaraw.cfgAlternatively if you want to use the plugin VisCustomTracker and to look only at tracker data , then use
trackerIguanaraw.py in the same directory.
Can I use a cfg file working with RAW data on RECO data?
Yes but you have to do some simple changes:
Replace
process.p = cms.Path( process.RawToDigi_woGCT * process.reconstructionCosmics)with
process.mypath = cms.Path(process.siPixelRecHits*process.siStripMatchedRecHits*process.TrackRefitter)
Add at the beginning:
process.load("RecoTracker.TrackProducer.RefitterWithMaterial_cff")
So which of the three programs is the easiest to use?
Frog. Fireworks comes second. And then iguana.
Which is the easiest to install?
Fireworks. The frog visualizer is also very easy to install and can be used to
look at events contained in .vis files available through Internet or accessible locally through afs.
Which is the fastest to scan many events?
Frog and Fireworks installed locally are the fastest and have almost the same speed. But Fireworks can be used only for RECO data.Iguana installed locally with enough resources can , in principle , reach the same speed. In practice , since in most cases it needs some CMSSW reconstruction, the real speed can be ten times slower. This is the case for example of raw data.
Which is the best to bring out the most relevant features of an event at first sight ?
Frog uses in a clever way the 3D representation, animation and fast exclusion ,inclusion of parts of signal/detector. For this reason is the best for the casual (non expert) user. Fireworks has the cleanest layout. Iguana with a black initial window is easy to beat (-:) but puts the complete event at your hand. After you become expert on how to configure it and on which signal/detector use, you will prefer it to the other two.
Which is the best if you are on a slow network connection to Cern and have no CMSSW installed locally?
Then the only alternative is frog. Frog is also the only one that can,in principle, also be run from an Internet cafe'.
Which is the best to produce nice pictures for the press or for publishing?
Iguana.
Which is the best to filter events with arbitrary cuts?
fireworks
What other tasks can be performed only by iguana?
All visualization tasks where its integration with CMSSW software is crucial.
If you need to know exactly what each event contains. The other two programs are more easy to use because they shield you from the complexity of the event. This means for example that displaying what is in some exotic data collection is a click away in iguana, requires some work with the menus and some writing with fireworks, and can be pretty impossible in frog (because the collection isn't in the .vis file) needing some expert work on the producer.
Another use of iguana impossible with the other two is the building of integrated signal trackermaps by VisCustomTracker.
What is the minimum hardware I need to run them?
A graphics card that understands opengl and is fast enough is mandatory for all three programs. A memory size > 1Gb also is very important. Big screen resolution then helps getting the most from these three tools. Frog is the software that needs less resources. For this , it is the only one that can run also on a Internet cafe' computer (provided that it has an adequate graphics card).
On which platform can I run them?
All three run on Linux SLC4. The frog visualizer can also run on Windows and fireworks on MAC. As said before frog is the only one that can be run on a "internet cafe'" computer: i.e. it doesn't need any CMSSW software and can get event information though a web adress.
I am interested in the events of some run. How can I find the datasets where it is stored?
Go to dbs . Click on "DBS discovery page". Then write in the first textfield "find run where run=xxxx". Choose the dataset in the page that you get. Click on "details" to know the Lumi section of each file in a dataset. This number gives the order in which the events where read.
The number of events in a dataset can be obtained by a query like this:
find sum(file.numevents) where dataset like /Cosmics/Commissioning08-PromptReco-v2/RECO and run=68021Once you have chosen the dataset , check that it is really present on castor by using the command:
rfdir /castor/cern.ch/cms/dataset_name
I am interested in a specific event of some run. How can I find the datasets where it is stored?
It seems that the information about first-last event in a dataset isn't written anywhere. The datasets are also listed on dbs in random order. The ordered list can be obtained if you sort them by lumi section number.This doesn't seem to be done automatically but ,once you know the dasaset name, you can have a list containing a line for each file name with the lumi sector. The search to be written in the textfield is:
find lumi,file where run=XXXXX AND dataset=dataset_name(for example:/Cosmics/Commissioning08-v1/RAW)Using this list you have to reason in this way. I search event 38099. The mean number of events in each dataset seem to be 30000 (this is reported in details list) so it should be in the file with lumi=2.
The web interface of dbs is very slooow:there is a way to get quick answers?
Yes. Just go in a directory where CMSSW is installed and give the usual "eval ...." . At this point a command "dbs" is defined that allows you to make searches by a command line. Here is an example that can be used as template for any request:
dbs --search --query "find lumi,file,file.numevents where dataset = */FEVT and run=122294 " --url=http://cmsdbsprod.cern.ch/cms_dbs_caf_analysis_01/servlet/DBSServlet
How I can find the correct global tag to be used with a specific run?
Most of the CMSSW cfg files used by iguana and frog have the following line:
process.GlobalTag.globaltag = "CRUZET4_V6P::All"where the magic word in the " " is the famous global tag. The list of spells can be reached from here and depends on CMSSW release and run number among other things. For example , for the normal offline use, for a CRAFT run you would use the global tag "CRAFT_VxP::All" with x the highest available when you run the program.
How I can know the correct magnetic field for a run?
Look this service.
How I can find the DQM summary plots of a run?
Use the DQM gui. More detailed plots can be examined by giving the following command on lxplus (in a directory CMSSW enabled):
root /afs/cern.ch/cms/CAF/CMSCOMM/COMM_DQM/data/Cosmics/068/021/DQM_V0001_R000068021__Cosmics__Commissioning08-PromptReco-v2__RECO.root(in this example we looked at run 68021 but it is quite obvious how to change run:the directory contains only 1 file and it is enough to give TAB to select it).
How can I know if a run is good?
Use the list of all runs with GOOD/BAD rating for all subdetectors.. Also this.
How can I plot some quantity that can be computed from DQM histograms for each run to see how it changes?
Luca Malgeri in a post has provided a recipe based on a python program that uses a cfg file and a list of runs. Just run it (in a CMSSW enabled directory on lxplus) giving:
python DQMscan.pyand then open in a browser dqcosm.html that should look like this
How I can know if during (a long run) there was some important change of working conditions?
There is this study by Andrea Venturi on SiStrip
There is a way to display only events with tracks?
The best way to access only events with tracks is to access skims. You can find them on dbs doing a search like this:
find dataset where dataset=/Cosmics/*Pointing*You get in this way a list of RECO datasets . If you are interested in a specific run you should select "Run info" and then the Run that interests you.
There is a way to display only events filtered by an arbitrary cut?
Yes, fireworks allow you to write arbitrary cuts in CINT and display only events that obey the cuts. For example , if in the "Filter" text field you write:
recoTracks_ctfWithMaterialTracksP5__Rec.@obj.size()>1 | SiStripClusteredmNewDetSetVector_siStripClusters__Rec.obj.@m_data.size()>40you will get displayed only events with more than one track or more than 40 tracker modules hit. For other examples of cuts just look this : List of useful commands to explore dataset
The same result is obtained in frog by running the Analyzer with a CMSSW filter. A CMSSW filter should in principle work also for Iguana but I was ,until now, unable to use it.
With iguana and frog you can use also another simple trick: give a list of events in the cfg file. This is an example on the way how you do it:
eventsToProcess = cms.untracked.VEventID('68021:15254','68021:29485','68021:83753','68021:89755','68021:113003','68021:365284','68021:369973','68021:365284'),
This list could be created automatically by a root macro where you apply the cut.
How can I access the image server on cmstac05 outside Cern?
I suppose that you have root access to your Linux box outside Cern. Access as root /etc/hosts/ adding the line
127.0.0.1 cmstac05.cern.chThen you should open the following terminal on lxplus:
ssh -l your_username -L 80:cmstac05.cern.ch:80 lxplus.cern.ch
What if I am on Windows and I want to access cmstac05 outside Cern?
There is a way to know how many events with tracks are in a specific dataset?
The easiest way is:
cd /tmp/Your_user_name/ (or other directory with enough space) scramv1 p CMSSW CMSSW_2_1_10 (a release compatible with the dataset you want to inspect) cd CMSSW_2_1_10/src/ root dataset_name (if in castor then add rfio:/castor/cern.ch/cms at beginning) TBrowser b (select root files) (select dataset) (select Events) (select track collection) (select obj) (select size) you should get the histogram of number of tracks per event in collectionThe same trick can be used to find if the sample has some other peculiar feature. For example events with big number of clusters. But keep in mind that root doesn't seem to work well with rfio: datasets. If it starts crashing for every histogram you request, then copy first the dataset locally on /tmp/ using
rfcp rfio_dataset /tmp/and then work on the local copy.
There is a way to know which are the events with highest number of tracks?
Proceed like we did before to plot the number of tracks for example in collection ctfWithMaterialTracks. Let's suppose you see there are 3 events with number of tracks > 20. Give in root the following command:
Events->Scan("recoTracks_ctfWithMaterialTracksP5__Rec.@obj.size():EventAuxiliary.id_.event_","recoTracks_ctfWithMaterialTracksP5__Rec.@obj.size()>20")
You should get a list like this:
*********************************** * Row * recoTrack* eventAuxi* *********************************** * 9107 * 22* 212750* ....9107 is the number of event in the dataset. 212750 the event number in run.
List of useful commands to explore dataset ;directory containg sample macros
If you want to save with Events->Scan lists of events, then use
Events->SetScanField(0); Events->Scan(...);>eventlist.txt
How can I get an ascii file with all castor files of a run ordered by lumi sector?
On dbs do the following search:
find lumi,file where run=68021 and dataset=/Cosmics/Commissioning08-PromptReco-v2/RECOSelect "all" then cut the result and paste it creating a file "filelist.txt". Now (on linux) give the command:
cat filelist.txt | sort -n | cut -d , -f 2 > sfilelist.txtsfilelist.txt contains the sorted list of files.
How can build an ntuple containing for each event of a run some very basic information like event number , LS, number of digi, number of clusters, number of tracks, etc: so I can quickly have a list of possibly interesting events?
How can I move events around if I don't have access on my graphics computer to castor?
Normally you have to do this in two steps:
ssh lxplus -l your_username cd /tmp/your_username/ rfcp /castor/cern.ch/cms/dataset_name . scp dataset_name your_username@your_computername:/your_datadirectory/If you want to copy only a few events , then :
cd CMSSW_ ../src/
eval ...
edit the following cfg file as copy.py:
import FWCore.ParameterSet.Config as cms
process = cms.Process("COPY")
process.maxEvents = cms.untracked.PSet( input = cms.untracked.int32(10) )
process.source = cms.Source("PoolSource",
fileNames = cms.untracked.vstring("/store/relval/CMSSW_2_1_6/RelValTTbar/GEN-SIM-DIGI-RAW-HLTDEBUG-RECO/IDEAL_V9_v1/0000/00B3662C-DA78-DD11-88E6-001617C3B70E.root")
)
process.copyAll = cms.OutputModule("PoolOutputModule", fileName = cms.untracked.string("reco.root") )
process.printEventNumber = cms.EDAnalyzer("AsciiOutputModule")
process.out = cms.EndPath(process.copyAll + process.printEventNumber)
cmsRun copy.py
scp outputfile your_username@your_computer:/your_datadirectory/
What is the easiest way to look at Montecarlo events?
As usual I would suggest to start with fireworks on lxplus. This will give you a rough idea on how the events look like.
ssh lxplus -l your_username cd /tmp/your_username/ /afs/cern.ch/cms/fireworks/cmsShow21/cmsShow rootfile
Ok done: now I want to look at the events in detail using iguana. I can run it locally on adequate machine. What I do?
Start in the usual way:
cd directory_with_enough_space scramv1 p CMSSW CMSSW_2_1_10 cd CMSSW_2_1_10/src iguana rootfile (next event)If you start playing with the menu on the left you see there is a problem:you can't show tracks. This is because now rechits are no more stored on disk. The thing to do is to run the track refitter before iguana. In practice:
stop iguana
mv iguana(Tab) iguana.py
vi iguana.py
(add the following lines at the beginning)
process.load("TrackingTools.TrackRefitter.TracksToTrajectories_cff")
process.load("RecoTracker.TrackProducer.RefitterWithMaterial_cff")
process.load("RecoTracker.TransientTrackingRecHit.TransientTrackingRecHitBuilderWithoutRefit_cfi")
process.TrackRefitter.src = 'generalTracks'
process.TrackRefitter.TrajectoryInEvent = True
(and this other as last line)
process.p = cms.Path(process.TrackRefitter)
iguana iguana.py
(using File menu create new rphi and rz window)
(maximize display then Window/tile)
(select parts of detector to display using the entries "Muon" "ECAL" "HCAL" "Tracker"at the end:don't use the Recodetector entries!)
(zoom and position the detector in 3D window)
(select next event)
(select signal that you want to display: i.e. generaltracks, Calotowers (all), muons, dt and csc segments)
(you may need also to change some parameters using the menu Configure/Edit : for example
"VisCaloEnergyScale" )
Can I use frog to look at MC data?
Yes. The procedure is the same used before for RAW data : replace only the file Frog_Analyzer_Cosmics_FromRaw.py with Frog_Analyzer.py
Where can I find MC events to test my cfg file?
Just look for most recent information at the Forum: RelVal Samples and Release Testing. Also you can try yourself searching on lxplus:
rfdir /castor/cern.ch/cms/store/relval/
How I can know if a new CMSSW release requests changes to my visualization cfg files?
Look in this directory(accessible only from Cern) . Starting from 229 I am trying to store some sample images for each release plus the cfg file used to produce them.
Bibliography: