Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce era modifier run3_nanoAOD_pre142X #46066

Merged
merged 2 commits into from
Sep 26, 2024

Conversation

hqucms
Copy link
Contributor

@hqucms hqucms commented Sep 19, 2024

PR description:

This PR introduces a new era modifier, run3_nanoAOD_pre142X, for running the NANO step from pre-14_2_X Run3 MiniAODs (e.g., 13_0_X and 14_0_X). This will allow for developments that move some sequences/products from the NANO step to the MINI step.

This resolves #46038.

PR validation:

runTheMatrix.py -w nano --ibeos -l 2500.101,2500.111,2500.201,2500.211

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 19, 2024

cms-bot internal usage

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46066/41856

  • There are other open Pull requests which might conflict with changes you have proposed:

@hqucms
Copy link
Contributor Author

hqucms commented Sep 19, 2024

enable nano

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @hqucms for master.

It involves the following packages:

  • Configuration/Eras (operations)
  • Configuration/PyReleaseValidation (upgrade, pdmv)
  • Configuration/StandardSequences (operations)
  • PhysicsTools/NanoAOD (xpog)

@AdrianoDee, @antoniovilela, @cmsbuild, @davidlange6, @fabiocos, @ftorrresd, @hqucms, @kskovpen, @mandrenguyen, @miquork, @rappoccio, @srimanob, @subirsarkar, @sunilUIET can you please review it and eventually sign? Thanks.
@AnnikaStein, @GiacomoSguazzoni, @JanFSchulte, @Martin-Grunewald, @VinInn, @VourMa, @dgulhan, @fabiocos, @felicepantaleo, @gpetruc, @makortel, @missirol, @mmusich, @mtosi, @rovere, @sameasy, @slomeo this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@hqucms
Copy link
Contributor Author

hqucms commented Sep 19, 2024

please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 32KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-62ea45/41650/summary.html
COMMIT: 9e0ba86
CMSSW: CMSSW_14_2_X_2024-09-19-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46066/41650/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 82 differences found in the comparisons
  • DQMHistoTests: Total files compared: 44
  • DQMHistoTests: Total histograms compared: 3331158
  • DQMHistoTests: Total failures: 5122
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3326016
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
  • Checked 193 log files, 163 edm output root files, 44 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially removed 678 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 21
  • DQMHistoTests: Total histograms compared: 55132
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 55132
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 20 files compared)
  • Checked 102 log files, 58 edm output root files, 21 DQM output files
  • TriggerResults: no differences found

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.001 2.841 2.841 0.000 ( +0.0% ) 3.13 3.10 +0.8% 6.678 6.037
2500.002 2.955 2.955 0.000 ( +0.0% ) 2.80 2.77 +1.1% 7.021 6.416
2500.003 2.901 2.901 0.000 ( +0.0% ) 2.89 2.89 -0.1% 6.997 6.391
2500.011 1.493 1.493 0.000 ( +0.0% ) 4.86 4.80 +1.4% 2.413 2.390
2500.012 1.959 1.959 0.000 ( +0.0% ) 2.89 2.91 -0.7% 2.608 2.533
2500.013 1.809 1.809 0.000 ( +0.0% ) 4.17 4.13 +1.0% 2.499 2.391
2500.021 0.022 0.022 0.000 ( +0.0% ) 0.94 0.94 -0.5% 2.209 2.200
2500.022 0.022 0.022 0.000 ( +0.0% ) 0.92 0.90 +1.9% 2.228 2.199
2500.023 0.022 0.022 0.000 ( +0.0% ) 0.92 0.89 +2.6% 2.183 2.138
2500.024 0.022 0.022 0.000 ( +0.0% ) 0.69 0.69 -0.1% 2.324 2.308
2500.031 0.035 0.035 0.000 ( +0.0% ) 0.85 0.84 +1.8% 2.325 2.281
2500.032 0.036 0.036 0.000 ( +0.0% ) 0.88 0.83 +5.6% 2.265 2.228
2500.033 0.037 0.037 0.000 ( +0.0% ) 0.79 0.78 +1.2% 2.368 2.327
2500.034 0.036 0.036 0.000 ( +0.0% ) 0.80 0.80 +1.0% 2.348 2.309
2500.101 2.660 2.660 0.000 ( +0.0% ) 8.85 8.88 -0.4% 6.910 6.298
2500.111 1.342 1.342 0.000 ( +0.0% ) 20.53 20.07 +2.3% 2.229 2.227
2500.112 1.749 1.749 0.000 ( +0.0% ) 14.35 15.01 -4.4% 2.117 1.803
2500.131 0.747 0.747 0.000 ( +0.0% ) 18.63 18.26 +2.0% 1.513 1.416
2500.201 2.490 2.490 0.000 ( +0.0% ) 7.62 7.48 +1.9% 6.150 5.830
2500.211 1.612 1.612 0.000 ( +0.0% ) 17.72 17.17 +3.2% 2.113 1.823
2500.212 2.055 2.055 0.000 ( +0.0% ) 14.11 13.94 +1.2% 2.142 1.857
2500.221 2.003 2.003 0.000 ( +0.0% ) 7.78 7.71 +1.0% 2.299 1.970
2500.222 3.222 3.222 0.000 ( +0.0% ) 7.39 7.44 -0.7% 2.353 2.063
2500.223 8.909 8.909 0.000 ( +0.0% ) 2.65 2.60 +2.1% 2.364 2.315
2500.224 5.680 5.680 0.000 ( +0.0% ) 0.61 0.60 +1.0% 2.448 2.385
2500.225 5.698 5.698 0.000 ( +0.0% ) 0.58 0.58 +1.0% 2.439 2.429
2500.226 2.988 2.988 0.000 ( +0.0% ) 6.91 7.34 -5.9% 2.303 2.297
2500.227 1.437 1.437 0.000 ( +0.0% ) 11.91 11.70 +1.8% 1.345 1.321
2500.231 1.403 1.403 0.000 ( +0.0% ) 13.80 13.57 +1.7% 1.952 1.935
2500.232 2.260 2.260 0.000 ( +0.0% ) 13.29 13.47 -1.3% 2.008 2.030
2500.233 4.707 4.707 0.000 ( +0.0% ) 4.31 4.06 +6.1% 2.037 2.019
2500.234 3.415 3.415 0.000 ( +0.0% ) 0.76 0.75 +1.3% 2.081 2.050
2500.235 3.426 3.426 0.000 ( +0.0% ) 0.74 0.74 -0.0% 2.108 2.110
2500.236 2.107 2.107 0.000 ( +0.0% ) 14.02 13.54 +3.6% 2.058 2.012
2500.237 1.016 1.016 0.000 ( +0.0% ) 17.00 16.67 +2.0% 1.351 1.346
2500.241 9.404 9.404 0.000 ( +0.0% ) 3.72 3.66 +1.6% 1.800 1.774
2500.242 10.331 10.331 0.000 ( +0.0% ) 0.92 0.90 +2.4% 1.641 1.610
2500.243 2.712 2.712 0.000 ( +0.0% ) 8.58 8.42 +1.9% 1.077 1.075
2500.244 485.976 485.976 0.000 ( +0.0% ) 0.57 0.55 +4.3% 1.692 1.606
2500.245 823.202 823.202 0.000 ( +0.0% ) 0.75 0.74 +1.9% 1.687 1.593
2500.901 1.777 1.777 0.000 ( +0.0% ) 21.63 20.88 +3.6% 1.802 1.766
2500.902 1.626 1.626 0.000 ( +0.0% ) 22.47 20.88 +7.6% 1.757 1.725
2500.911 13.995 13.995 0.000 ( +0.0% ) 3.88 2.93 +32.5% 1.093 1.088
2500.912 0.150 0.729 -0.579 ( -79.4% ) 0.90 1.29 -30.0% 0.978 0.974
2500.913 0.110 0.110 0.000 ( +0.0% ) 1.19 1.11 +7.2% 0.977 0.979

@hqucms
Copy link
Contributor Author

hqucms commented Sep 20, 2024

+1

@hqucms
Copy link
Contributor Author

hqucms commented Sep 23, 2024

@cms-sw/pdmv-l2 @cms-sw/upgrade-l2 Could you please review and sign? We have a few PRs pending on this to be merged first.

@Moanwar
Copy link
Contributor

Moanwar commented Sep 24, 2024

+Upgrade

@hqucms
Copy link
Contributor Author

hqucms commented Sep 26, 2024

A gentle ping @cms-sw/pdmv-l2 @cms-sw/operations-l2 -- this is blocking #46112 and a few more developments in the pipeline for NanoAOD v15. Thanks!

@AdrianoDee
Copy link
Contributor

Hi @hqucms dummy question. While I understand the need of the new modifier, I don't understand why it becomes the default one for the current cycle NANO wfs right?. Shouldn't it be differentiated (as is done for the other cycles)?

@hqucms
Copy link
Contributor Author

hqucms commented Sep 26, 2024

Hi @hqucms dummy question. While I understand the need of the new modifier, I don't understand why it becomes the default one for the current cycle NANO wfs right?. Shouldn't it be differentiated (as is done for the other cycles)?

Hi @AdrianoDee -- It's not the default if the MINI input is also produced in the current (14_2_X) cycle. It's only for reading MiniAODs from previous cycles , eg. 140X for 2024, 130X for 2022/2023, etc.

@AdrianoDee
Copy link
Contributor

+pdmv

@antoniovilela
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

@cmsbuild cmsbuild merged commit a291790 into cms-sw:master Sep 26, 2024
13 checks passed
@hqucms hqucms deleted the xpog/nano-era-pre142X branch December 17, 2024 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New modifier(s) for NanoAOD v15?
5 participants