Activate the singularity image (to be done every time before running code)
singularity run /cvmfs/unpacked.cern.ch/gitlab-registry.cern.ch/bendavid/cmswmassdocker/wmassdevrolling\:latestActivate git Large File Storage (only need to do this once for a given user/home directory)
git lfs install
Get the code (after forking from the central WMass repository)
MY_GIT_USER=$(git config user.github)
git clone --recurse-submodules [email protected]:$MY_GIT_USER/WRemnants.git
cd WRemnants/
git remote add upstream [email protected]:WMass/WRemnants.gitGet updates from the central repository (and main branch)
git pull --recurse-submodules upstream main
git push origin mainGet combinetf. Need to run cmssw-cc7 (outside of the other singularity image) to work in a special centos7 environment, which allows you to work with CMSSW. If you plan to contribute to the combinetf code, you may first fork from: https://github.com/bendavid/HiggsAnalysis-CombinedLimit
cmssw-cc7
cd /some/path/to/download/code/
export SCRAM_ARCH="slc7_amd64_gcc700"
cmsrel CMSSW_10_6_19_patch2
cd CMSSW_10_6_19_patch2/src/
cmsenv
git clone -o bendavid -b tensorflowfit [email protected]:bendavid/HiggsAnalysis-CombinedLimit.git HiggsAnalysis/CombinedLimit
cd HiggsAnalysis/CombinedLimit
scram b -j 8
#
# if everything worked fine, folder scripts/ contains "combineCards.py, combinetf.py, commentUncerts.py, pruneUncerts.py, text2hdf5.py, text2workspace.py"
# the reference branch is bendavid/tensorflowfit
# so your current local branch should also be tensorflowfit
#
# optional for developments, if you have your own remote fork
git remote add origin [email protected]:<YOUR_GITHUB_USER>/HiggsAnalysis-CombinedLimit.git
git checkout -b myBranch
git push origin myBranch
To run the fit with combinetf
cmssw-cc7
cd /path/to/CMSSW_10_6_19_patch2/src/HiggsAnalysis/CombinedLimit/
cmsenv
cd /wherever/you/like/
<commands to run fit> # e.g. using WRemnants/scripts/combine/fitManager.py
Guidelines
- Use camel case practice for command line arguments and avoid the "dest" keyword.
- Use snake case practice for function names.
- Class names should start with capital letters.
- When making a new PR, it should target only one subject at a time. This makes it more easy to validate and the integration faster. PR on top of other PR are ok when it is noted in the description, e.g. this PR is on top of PR XYZ.
Source the setup script. It will create some environment variables to ease access to some folders:
- WREM_BASE: it points to ./WRemnants/ where all the code is
- COMBINE_STUDIES: folder with codes to create datacards and root files for combine, and run the fit
- PLOTS: folder with some scripts for plots and dedicated studies for analysis
source WRemnants/setup.shMake histograms (only nominal and mass variations for now, systematics are being developed)
/usr/bin/time -v python scripts/histmakers/mw_with_mu_eta_pt.py -o outputFolder/ --theoryAgnostic --noAuxiliaryHistograms
Prepare datacards and root files with TH2 (stat-only for now)
/usr/bin/time -v python scripts/combine/setupCombine.py -i outputFolder/mw_with_mu_eta_pt_scetlib_dyturboCorr.hdf5 -o outputFolder/ --absolutePathInCard --theoryAgnostic
To remove the backgrounds and run signal only one can add --excludeProcGroups Top Diboson Fake Zmumu DYlowMass Ztautau Wtaunu BkgWmunu
Run the fit (for charge combination)
python WRemnants/scripts/combine/fitManager.py -i outputFolder/WMass_pt_eta_statOnly/ --skip-fit-data --theoryAgnostic --comb
Make histograms (this has all systematics too unlike the standard theory agnostic setup)
/usr/bin/time -v python scripts/histmakers/mw_with_mu_eta_pt.py -o outputFolder/ --theoryAgnostic --poiAsNoi
Prepare datacards and root files with TH2
/usr/bin/time -v python scripts/combine/setupCombine.py -i outputFolder/mw_with_mu_eta_pt_scetlib_dyturboCorr.hdf5 -o outputFolder/ --absolutePathInCard --theoryAgnostic --poiAsNoi --priorNormXsec 0.5
To remove the backgrounds and run signal only one can add --filterProcGroups Wmunu
Run the fit (for charge combination). Note that it is the same command as the traditional analysis, without --theoryAgnostic
python WRemnants/scripts/combine/fitManager.py -i outputFolder/WMass_pt_eta/ --skip-fit-data --comb
Make histograms for WMass (for Wlike the script is mz_wlike_with_mu_eta_pt.py).
python WRemnants/scripts/histmakers/mw_with_mu_eta_pt.py -o outputFolder/
More options are loaded from WRemnants/utilities/common.py
Make the datacards for single charges and prepare the TH2 histograms for combinetf.
python WRemnants/scripts/combine/setupCombine.py -i outputFolder/mw_with_mu_eta_pt_scetlib_dyturboCorr.hdf5 -o outputFolder/
The input file is the output of the previous step. The default path specified with -o is the local folder. A subfolder with name identifying the specific analysis (e.g. WMass_pt_eta/) is automatically created inside it. Some options may add tags to the folder name: for example, using --doStatOnly will call the folder WMass_pt_eta_statOnly/. Can use --absolutePathInCard to write absolute path for files in the datacard, so to allow one to run the fit from any location.
Combine the datacards for single charges and run the fit (Asimov only)
python WRemnants/scripts/combine/fitManager.py -i outputFolder/WMass_pt_eta/ --comb --skip-fit-data
Run the fits for single charges (Asimov only). These can be produced in the same output folder as the combination, since a postfix is automatically appended to the output card and fit result files.
python WRemnants/scripts/combine/fitManager.py -i outputFolder/WMass_pt_eta/ --fit-single-charge --skip-fit-data [-c <plus|minus|both>]
NOTE:
- to run fitManager.py one has to set a Centos 7 environment with cmssw-cc7. Then, one has to activate cmsenv from the folder where combine is installed (once the environment is set one can keep working from inside WRemnants).
- Each script has tons of options, to customize a gazillion of things, it's simpler to learn them by asking an expert rather that having an incomplete summary here (developments happen faster than documentation anyway).
There are many scripts to do every kind of plotting, and different people may have their own ones. We'll try to put a minimal list with examples here ASAP.
Plot Wmass histograms from hdf5 file (from Wmass histmaker) in the 4 iso-MT regions (can choose only some). It also makes some plots for fakes depending on the chosen region. It is also possible to select some specific processes to put in the plots.
python scripts/analysisTools/tests/testShapesIsoMtRegions.py mw_with_mu_eta_pt_scetlib_dyturboCorr.hdf5 outputFolder/ [--isoMtRegion 0 1 2 3]
Plot prefit shapes (requires root file from setupCombine.py as input)
python scripts/analysisTools/w_mass_13TeV/plotPrefitTemplatesWRemnants.py WMassCombineInput.root outputFolder/ [-l 16.8] [--pseudodata <pseudodataHistName>] [--wlike]
Make study of fakes for mW analysis, checking mT dependence, with or without dphi cut (see example inside the script for more options). Even if the histmaker was run with the dphi cut, the script uses a dedicated histograms mTStudyForFakes created before that cut, and with dphi in one axis.
python scripts/analysisTools/tests/testFakesVsMt.py mw_with_mu_eta_pt_scetlib_dyturboCorr.hdf5 outputFolder/ --rebinx 4 --rebiny 2 --mtBinEdges "0,5,10,15,20,25,30,35,40,45,50,55,60,65" --mtNominalRange "0,40" --mtFitRange "0,40" --fitPolDegree 1 --integralMtMethod sideband --maxPt 50 --met deepMet [--dphiMuonMetCut 0.25]
Make quick plots of any 1D distribution produced with any histmaker
python scripts/analysisTools/tests/testPlots1D.py mz_wlike_with_mu_eta_pt_scetlib_dyturboCorr.hdf5 outputFolder/ --plot transverseMass_uncorr transverseMass -x "Uncorrected Wlike m_{T} (GeV)" "Corrected Wlike m_{T} (GeV)"
Make plot with mW impacts from a single fit result
python scripts/analysisTools/w_mass_13TeV/makeImpactsOnMW.py fitresults_123456789.root -o outputFolder/ --scaleToMeV --showTotal -x ".*eff_(stat|syst)_" [--postfix plotNamePostfix]
Make plot with mW impacts comparing two fit results
python scripts/analysisTools/w_mass_13TeV/makeImpactsOnMW.py fitresults_123456789.root -o outputFolder/ --scaleToMeV --showTotal --compareFile fitresults_123456789_toCompare.root --printAltVal --legendEntries "Nominal" "Alternate" -x ".*eff_(stat|syst)_" [--postfix plotNamePostfix]
Print impacts without plotting (no need to specify output folder)
python w_mass_13TeV/makeImpactsOnMW.py fitresults_123456789.root --scaleToMeV --showTotal --justPrint
Make W MC efficiencies for trigger and isolation (needed for anti-iso and anti-trigger SF)
/usr/bin/time -v python scripts/histmakers/mw_with_mu_eta_pt.py -o outputFolder/ --makeMCefficiency --onlyMainHistograms --noAuxiliaryHistograms --noScaleFactors --muonCorrMC none -p WmunuMCeffi_noSF_muonCorrMCnone --filterProcs Wmunu --dataPath root://eoscms.cern.ch//store/cmst3/group/wmass/w-mass-13TeV/NanoAOD/ -v 4 --maxFiles -1
python scripts/analysisTools/w_mass_13TeV/makeWMCefficiency3D.py /path/to/file.hdf5 /path/for/plots/makeWMCefficiency3D/ --rebinUt 2
Then, run 2D smoothing (has to manually edit the default input files inside for now, see other options inside too). Option --extended was used to select SF computed in a larger ut range, but now this might become the default (to be updated)
python scripts/analysisTools/w_mass_13TeV/run2Dsmoothing.py /path/for/plots/test2Dsmoothing/