Add track-based isolation, recHit energy sum to IsolatedTrack class

[czkaiweb]

Isolated track PR:

PR description:

1. track-based isolation and associated recHit energy sum added to IsolatedTrack class
2. exotical disappearing track cut added to extend dedxHitInfo for isolated track of interest

PR validation:

Validation done under CMSSW_10_2_14.
MINIAOD with additional IsolatedTrack information is created from AOD for:


Data 2018:
EGamma
SingleMu
MET
JetHT

MC 2017:
Drell-Yan

MC 2018:
WJets
TT
QCD50to80

DATA configure file:
cmsDriver,py REMINIAOD -s PAT --runUnscheduled --nThreads 4 --data --era Run2_2018,run2_miniAOD_devel --scenario pp --conditions 102X_dataRun2_Sep2018ABC_v2 --eventcontent MINIAOD --datatier MINIAOD --filein file:pippo.root -n -1 --python_filename=IsolatedTrack_miniAOD_DATA2018XXX.py --no_exec

MC configure file:
cmsDriver.py step1 --filein file:pippo.root --fileout file:step1_PAT.root --mc --eventcontent MINIAODSIM --runUnscheduled --datatier MINIAODSIM --conditions 102X_upgrade2018_realistic_v15 --step PAT --nThreads 4 --geometry DB:Extended --era Run2_2018 --python_filename IsolatedTrack_miniAOD_MC2018XXX.py --no_exec --customise Configuration/DataProcessing/Utils.addMonitoring -n -1

Change of Event Size is measured:
https://docs.google.com/spreadsheets/d/177RQLW9lWvk18EtzNt4-wqAfgLVffC4aGCh8HJZLwTg/edit?usp=sharing

Number of events passing slimming :
https://docs.google.com/spreadsheets/d/1PwBgbf29jjE-NzIr0HFQrYjZZGkBZRl_npwaW9w_WDM/edit?usp=sharing

Before submitting your pull requests, make sure you followed this checklist:

• verify that the PR is really intended for the chosen branch:
  Checked
• verify that changes follow CMS Naming, Coding, And Style Rules
  Checked
• verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions
  Failed to run over Workflow in the link due to file access error, e.g.
  """Failed to open the file 'root://cmsxrootd.fnal.gov//store/relval/CMSSW_9_2_2/RelValProdTTbar_13/AODSIM/91X_mcRun2_asymptotic_v3-v1/10000/EEB99F74-DA4D-E711-A41C-0025905A48F2.root'"""
  PR still submitted for further recommendation regarding workflow test

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31399/18248

• This PR adds an extra 44KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File DataFormats/PatCandidates/src/classes_def_objects.xml modified in PR(s): remove vertex type from PFCandidate (addresses #30895) #31298

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31399/18248/code-format.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31399/18248/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[perrotta]

@czkaiweb , unless you apply the code format fixes as explained above in #31399 (comment) this PR cannot start automatic tests, and therefore it cannot get reviewed for approval

[perrotta]

assign xpog

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31399/18299

• This PR adds an extra 36KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File DataFormats/PatCandidates/src/classes_def_objects.xml modified in PR(s): remove vertex type from PFCandidate (addresses #30895) #31298

Code check has found code style and quality issues which could be resolved by applying following patch(s)

• code-format:
  https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31399/18299/code-format.patch
  e.g. curl https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31399/18299/code-format.patch | patch -p1
  You can also run scram build code-format to apply code format directly

[cmsbuild]

The code-checks are being triggered in jenkins.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31399/18302

• This PR adds an extra 44KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File DataFormats/PatCandidates/src/classes_def_objects.xml modified in PR(s): remove vertex type from PFCandidate (addresses #30895) #31298

[cmsbuild]

New categories assigned: xpog

@fgolf,@mariadalfonso,@gouskos,@peruzzim you have been requested to review this Pull request/Issue and eventually sign? Thanks

[carriganm95]

Hi @perrotta @gouskos @czkaiweb I wanted to post some plots following up on our last conversation with the Cross-POG on this topic. We have looked at track isolation and calo rec hits in miniAOD vs AOD since our last presentation. We found that track isolation in miniAOD is good enough for us but the available rec hit collections are not. In the attached plots you can see in muon and electron samples from 2017 we would have allowed significantly more background into our analysis. Please let me know if there are any questions and what we can do to proceed. Thanks
EcaloPRSlides.pdf

[gouskos]

Hi @carriganm95 - thank you for the update. I have the following questions:
(a) What is the size increase when storing only the CaloRecHit sum. It would be useful to know which type of RecHits are not included in the RecHit collections in miniAOD. As it is currently the PR, it stores two floats [ECAL and HCAL sums] for all isolated tracks - maybe is more size-efficient to update the RecHit collection [e.g., for tracks above some pT threshold]
(b) From your slides only muons are affected [within the stats you have]. Can you test with a larger sample to have a better estimation?
(c) Did you investigate further the possibility of developing a skim that outputs AOD ? e.g., among the lines of your selection in slide 1?
(d) Finally, do you have any update on whatever other analyses in EXO, SUS started already exploring this ? In the last update at CrossPOG there was no commitment from other analyses to use it

Hi @perrotta @gouskos @czkaiweb I wanted to post some plots following up on our last conversation with the Cross-POG on this topic. We have looked at track isolation and calo rec hits in miniAOD vs AOD since our last presentation. We found that track isolation in miniAOD is good enough for us but the available rec hit collections are not. In the attached plots you can see in muon and electron samples from 2017 we would have allowed significantly more background into our analysis. Please let me know if there are any questions and what we can do to proceed. Thanks EcaloPRSlides.pdf

[jpata]

Is this PR coming back to life now after a year?
Once the XPOG questions are addressed, if a reco review and test is expected, please remove the Draft label.

[gouskos]

Hi @carriganm95 - I have one more question. The RecHits not included in the Reduced collection in MiniAOD should primarily include noisy crystals. I believe using Energy from Noisy crystals to veto event could hurt the signal efficiency. On slides 2 & 3, can you please super impose the distribution for a couple signal models?

[carriganm95]

A. Attached is a table made by Kai Wei looking at the size increase in several samples when we add in one double for ECAL+HCAL in isolatedTracks, and at adding hits to the rec hit collections. As you can see from the table adding the rec hits is a huge size increase, while adding the one double is much more compact.

B. Unfortunately there are no larger samples of singleElectron data produced after CMSSW_11 available (aside from some RAW). We have used CMSSW_11_2 to produce the samples because the rec hit collections for miniAOD have been updated (sometime early CMSSW_11). Therefore any other older sample will have fewer rec hits in common between AOD and miniAOD. But as the plot I previously sent shows the energy in AOD is larger, so we can expect more background, it is just hard to estimate how much.

C. We are pursuing this option and have informed exo LL of our intentions. These skims will be metMinimalSkim (3.2TB), basic selection (320GB), single electron (8.9TB), single muon (15.3 TB), and Tau (28.3 GB).

D. We are trying to recirculate the idea among the exo community by giving presentations on the idea at the exo-LL meeting.

I have send the plots that you asked for including signal. We have been aware that this would increase our signal acceptance but this does not help us more than hurt us. Our analysis has very low backgrounds so the few added lepton backgrounds is worse for our sensitivity.

Ecalo Size Increase - Sheet1.pdf

[jpata]

kind ping on this

[gouskos]

Thank you

A. Attached is a table made by Kai Wei looking at the size increase in several samples when we add in one double for ECAL+HCAL in isolatedTracks, and at adding hits to the rec hit collections. As you can see from the table adding the rec hits is a huge size increase, while adding the one double is much more compact.

For the uncompressed (compressed) the size increase / collection is <1% (~3%)

B. Unfortunately there are no larger samples of singleElectron data produced after CMSSW_11 available (aside from some RAW). We have used CMSSW_11_2 to produce the samples because the rec hit collections for miniAOD have been updated (sometime early CMSSW_11). Therefore any other older sample will have fewer rec hits in common between AOD and miniAOD. But as the plot I previously sent shows the energy in AOD is larger, so we can expect more background, it is just hard to estimate how much.

C. We are pursuing this option and have informed exo LL of our intentions. These skims will be metMinimalSkim (3.2TB), basic selection (320GB), single electron (8.9TB), single muon (15.3 TB), and Tau (28.3 GB).

Just to confirm. With these skims you are able to do the full the analysis, right? What are these sizes correspond to i.e., what amount of data?

D. We are trying to recirculate the idea among the exo community by giving presentations on the idea at the exo-LL meeting.

I have send the plots that you asked for including signal. We have been aware that this would increase our signal acceptance but this does not help us more than hurt us. Our analysis has very low backgrounds so the few added lepton backgrounds is worse for our sensitivity.

Ecalo Size Increase - Sheet1.pdf

[carriganm95]

@gouskos Attached is a list of all datasets used in our run 2 analysis and the corresponding sizes.
Run2DatasetSizes.pdf

[jpata]

Is this expected to be non-draft and ready to be reviewed / taken forward?
Otherwise, if this is still in discussion, reco will -1 this in the next days.

[perrotta]

I realized just now that the "-1" I wrote as reco reviewer long time ago was automatically moved from "reconstruction rejected" to "orp rejected" when I changed my hat. I have therefore commented out that old entry of mine in this thread, so that this draft PR is not "orp rejected" any more. I let the current reconstruction conveners continue dealing with it.

[gouskos]

@jpata @perrotta Please give us a couple of days to conclude here.

[jpata]

kind ping

[gouskos]

kind ping

Hi @jpata we discussed with the EXO conveners (@lowette @zdemirag ) indeed this is the only analysis that needs this PR. Since the analysis can be carried out with skims (@carriganm95 please confirm) the idea is to not propagate this change. The EXO conveners have probably already discussed this with the authors and the EXO-CrossPOG contacts.

[jpata]

Thanks for confirming! Should this PR be closed then?

[gouskos]

Thanks for confirming! Should this PR be closed then?

I believe so - and I guess if something changes we can re-open it.
Thanks for your patience!

[jpata]

@cmsbuild please close

[carriganm95]

Hi @jpata @gouskos sorry we had not discussed this with the exo conveners. Even if we use skims we do still need our "_exoDisappearingTrackCut" in saveDeDxHitInfoCut in the slimming/isolatedTracks_cfi.py file. Would you prefer we submit a new PR or just edit this one?

[gouskos]

Hi @carriganm95 I think a separate PR would be better.
Of course, this should be enabled only when producing the skim