Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to run hlt p2 timing script for PRs #2240

Merged
merged 20 commits into from
May 24, 2024
Merged

Conversation

iarspider
Copy link
Contributor

No description provided.

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @iarspider for branch master.

@cmsbuild, @aandvalenzuela, @iarspider, @smuzaffar can you please review it and eventually sign? Thanks.
@antoniovilela, @rappoccio, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented May 22, 2024

cms-bot internal usage

process_pr.py Outdated Show resolved Hide resolved
@smuzaffar
Copy link
Contributor

I do not see change in https://github.com/cms-sw/cms-bot/blob/master/pr_testing/test_multiple_prs.sh to run the new test

@cmsbuild
Copy link
Contributor

Pull request #2240 was updated.

@iarspider
Copy link
Contributor Author

And this is still not exactly right...

@cmsbuild
Copy link
Contributor

Pull request #2240 was updated.

@cmsbuild
Copy link
Contributor

Pull request #2240 was updated.

@smuzaffar
Copy link
Contributor

enable hlt_p2_timing

@smuzaffar
Copy link
Contributor

please test


# Do work
timeout $TIMEOUT ${CMSSW_CVMFS_PATH}/src/HLTrigger/Configuration/python/HLT_75e33/test/runHLTTiming.sh 2>&1 | tee $WORKSPACE/hlt-p2-timing.log
CHART_URL="/circles/web/piechart.php?data_name=pr-hlt-p2-timing&resource=time_thread&filter=${UPLOAD_UNIQ_ID}&dataset=${UPLOAD_UNIQ_ID}/Phase2Timing_resources"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pr-hlt-p2-timing will not work. We need to upload the json files in the same directory as IBs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why mix IB and PR results?

Copy link
Contributor Author

@iarspider iarspider May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I see, it's just a matter of adding a new symlink from PR testing area to /circles

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so that one can see both IB and PR tests in the drop down

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

@@ -1157,11 +1158,20 @@ if [ "X$BUILD_OK" = Xtrue -a "$RUN_TESTS" = "true" ]; then
DO_GPU_TESTS=true
mark_commit_status_all_prs 'unittests/gpu' 'pending' -u "${BUILD_URL}" -d "Waiting for tests to start"
fi
if [ $(echo ${ENABLE_BOT_TESTS} | tr ',' ' ' | tr ' ' '\n' | grep '^HLT_P2_TIMING$' | wc -l) -gt 0 ] ; then
if [ $(echo ${ARCHITECTURE} | grep "_amd64_" | wc -l) -gt 0 ] ; then
if [ -e ${CMSSW_CVMFS_PATH}/src/HLTrigger/Configuration/python/HLT_75e33/test/runHLTTiming.sh ]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please use $CMSSW_RELEASE_BASE instead of ${CMSSW_CVMFS_PATH} ( not that CMSSW_CVMFS_PATH is not accessible when this script is running. Output of this script gets deployed under CMSSW_CVMFS_PATH)

echo "HLT P2 timing: [chart](${CHART_URL})" > ${RESULTS_DIR}/hlt-p2-timing-report.res

mv WORKSPACE/Phase2Timing_resources*.json $WORKSPACE/testsResults
send_jenkins_artifacts $WORKSPACE/testsResults pr-hlt-p2-timing/${UPLOAD_UNIQ_ID}
Copy link
Contributor

@smuzaffar smuzaffar May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

upload it under hlt-p2-timing/<CMSSW_Version>+<RepoName><PRNUM>/$SCRAM_ARCH/${BUILD_NUMBER} e.g. hlt-p2-timing/CMSSW_14_1_X_2024-05-22-2300+cmssw44500/el8_amd64_gcc12/1234

@cmsbuild
Copy link
Contributor

Pull request #2240 was updated.

@smuzaffar
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

Pull request #2240 was updated.

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-50770a/39500/summary.html
COMMIT: 31eb67e
CMSSW: CMSSW_14_1_X_2024-05-23-1100/el8_amd64_gcc12
Additional Tests: HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/2240/39500/install.sh to create a dev area with all the needed externals and cmssw changes.

HLT P2 timing: chart

Comparison Summary

Summary:

  • You potentially removed 3 lines from the logs
  • Reco comparison results: 8 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3338862
  • DQMHistoTests: Total failures: 3
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3338839
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 202 log files, 165 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

@iarspider iarspider marked this pull request as ready for review May 24, 2024 07:21
@cmsbuild
Copy link
Contributor

Pull request #2240 was updated.

@smuzaffar
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

Pull request #2240 was updated.

@smuzaffar
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

Pull request #2240 was updated.

@cmsbuild
Copy link
Contributor

cmsbuild commented May 24, 2024

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-50770a/39509/summary.html
COMMIT: 8411dfd
CMSSW: CMSSW_14_1_X_2024-05-23-2300/el8_amd64_gcc12
Additional Tests: HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/2240/39509/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 12 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3338862
  • DQMHistoTests: Total failures: 6
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3338836
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 202 log files, 165 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

@smuzaffar
Copy link
Contributor

smuzaffar commented May 24, 2024

FYI @rovere , this PR allows to enable hlt_p2_timing tests via either enable hlt_p2_timing command or enable = hlt_p2_timing test parameter comment. Piechart for IB+PR are access via link in #2240 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants