Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update libffi to 3.4.2 #7276

Merged
merged 1 commit into from
Nov 4, 2021

Conversation

mrodozov
Copy link
Contributor

@mrodozov mrodozov commented Sep 7, 2021

@mrodozov
Copy link
Contributor Author

mrodozov commented Sep 7, 2021

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 7, 2021

A new Pull Request was created by @mrodozov (Mircho Rodozov) for branch IB/CMSSW_12_1_X/master.

@smuzaffar, @mrodozov, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @qliphy you are the release manager for this.
cms-bot commands are listed here

@mrodozov
Copy link
Contributor Author

mrodozov commented Sep 7, 2021

please test for slc7_aarch64_gcc9

@mrodozov
Copy link
Contributor Author

mrodozov commented Sep 7, 2021

please test for slc7_ppc64le_gcc9

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 8, 2021

-1

Failed Tests: UnitTests RelVals AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-875cf9/18371/summary.html
COMMIT: bb8fb33
CMSSW: CMSSW_12_1_X_2021-09-06-2300/slc7_ppc64le_gcc9
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-875cf9/18371/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-875cf9/18371/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test test_PrepareInputDb had ERRORS
---> test test_MpsWorkFlow had ERRORS
---> test testFWCoreUtilities had ERRORS
---> test TestHeterogeneousCoreSonicTritonProducerGPU had ERRORS
and more ...

RelVals

----- Begin Fatal Exception 08-Sep-2021 07:46:33 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named Pyquen_ZeemumuJets_pt10_2760GeV_cfi_GEN_SIM_PU.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file SimG4CMS/Calo/data/HFShowerLibrary_run3_v5.root anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/external/slc7_ppc64le_gcc9/data:/cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc9/cms/cmssw/CMSSW_12_1_X_2021-09-06-2300/src
Current directory is: /scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-relvals/runTheMatrix-results/312.0_Pyquen_ZeemumuJets_pt10_2760GeV_2021+Pyquen_ZeemumuJets_pt10_2760GeV_2021+DIGIHI2021MIX+RECOHI2021MIX+HARVESTHI2021PPRECO


At:
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(840): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(925): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(918): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Config.py(1092): _insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Config.py(1306): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 08-Sep-2021 07:46:36 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named TTbar_14TeV_TuneCP5_cfi_GEN_SIM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file SimG4CMS/Calo/data/HFShowerLibrary_run3_v5.root anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/external/slc7_ppc64le_gcc9/data:/cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc9/cms/cmssw/CMSSW_12_1_X_2021-09-06-2300/src
Current directory is: /scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-relvals/runTheMatrix-results/11634.0_TTbar_14TeV+2021+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA


At:
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(840): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(925): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(918): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Config.py(1092): _insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Config.py(1306): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 08-Sep-2021 07:46:38 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named TTbar_14TeV_TuneCP5_cfi_GEN_SIM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file SimG4CMS/Calo/data/HFShowerLibrary_run3_v5.root anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/external/slc7_ppc64le_gcc9/data:/cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc9/cms/cmssw/CMSSW_12_1_X_2021-09-06-2300/src
Current directory is: /scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-relvals/runTheMatrix-results/11634.911_TTbar_14TeV+2021_DD4hep+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA


At:
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(840): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(925): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(918): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Config.py(1092): _insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Config.py(1306): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

AddOn Tests

----- Begin Fatal Exception 08-Sep-2021 07:35:03 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named TTbar_Tauola_13TeV_TuneCUETP8M1_cfi_GEN_SIM_DIGI_L1_DIGI2RAW.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file SimG4CMS/Calo/data/HFShowerLibrary_run3_v5.root anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/external/slc7_ppc64le_gcc9/data:/cvmfs/cms-ib.cern.ch/week1/slc7_ppc64le_gcc9/cms/cmssw/CMSSW_12_1_X_2021-09-06-2300/src
Current directory is: /scratch/cmsbuild/jenkins_a/workspace/ib-run-pr-addon/addOnTests/hlt_mc_GRun


At:
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(840): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Mixins.py(377): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(925): insertContentsInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Types.py(918): insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Config.py(1092): _insertInto
  /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18371/CMSSW_12_1_X_2021-09-06-2300/python/FWCore/ParameterSet/Config.py(1306): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 08-Sep-2021 07:35:50 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling File::sysopen()
Exception Message:
Failed to open the file 'RelVal_Raw_GRun_MC.root'
   Additional Info:
      [a] Input file file:RelVal_Raw_GRun_MC.root could not be opened.
      [b] open() failed with system error 'No such file or directory' (error code 2)
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 08-Sep-2021 07:38:55 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling File::sysopen()
Exception Message:
Failed to open the file 'RelVal_Raw_GRun_MC.root'
   Additional Info:
      [a] Input file file:RelVal_Raw_GRun_MC.root could not be opened.
      [b] open() failed with system error 'No such file or directory' (error code 2)
----- End Fatal Exception -------------------------------------------------
Expand to see more addon errors ...

@smuzaffar
Copy link
Contributor

@mrodozov , it is not libtiff ... right? please update the PR title and commit message

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 8, 2021

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-875cf9/18370/summary.html
COMMIT: bb8fb33
CMSSW: CMSSW_12_1_X_2021-09-07-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18370/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 1307 differences found in the comparisons
  • DQMHistoTests: Total files compared: 39
  • DQMHistoTests: Total histograms compared: 3001001
  • DQMHistoTests: Total failures: 3682
  • DQMHistoTests: Total nulls: 20
  • DQMHistoTests: Total successes: 2997277
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 45.699 KiB( 38 files compared)
  • DQMHistoSizes: changed ( 140.53 ): 44.531 KiB Hcal/DigiRunHarvesting
  • DQMHistoSizes: changed ( 140.53 ): 1.172 KiB RPC/DCSInfo
  • DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
  • Checked 165 log files, 37 edm output root files, 39 DQM output files
  • TriggerResults: no differences found

@mrodozov mrodozov changed the title Update libtiff to 3.4.2 Update libffi to 3.4.2 Sep 8, 2021
@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 8, 2021

Pull request #7276 was updated.

@mrodozov
Copy link
Contributor Author

mrodozov commented Sep 8, 2021

@mrodozov , it is not libtiff ... right? please update the PR title and commit message

yes, sorry about that it's strange I named the branch properly and malapropped the commit message

@smuzaffar
Copy link
Contributor

test parameters:

  • addpkg = Alignment,CondCore

@smuzaffar
Copy link
Contributor

please test for slc7_ppc64le_gcc9

@smuzaffar
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 9, 2021

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-875cf9/18440/summary.html
COMMIT: 76f9fdc
CMSSW: CMSSW_12_1_X_2021-09-08-2300/slc7_ppc64le_gcc9
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18440/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-875cf9/18440/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-875cf9/18440/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test test_PrepareInputDb had ERRORS
---> test test_MpsWorkFlow had ERRORS
---> test testFWCoreUtilities had ERRORS
---> test GeneratorInterfaceEvtGenInterfaceTest had ERRORS
and more ...

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 9, 2021

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-875cf9/18441/summary.html
COMMIT: 76f9fdc
CMSSW: CMSSW_12_1_X_2021-09-08-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7276/18441/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-875cf9/18441/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-875cf9/18441/git-merge-result

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 39
  • DQMHistoTests: Total histograms compared: 3001001
  • DQMHistoTests: Total failures: 6
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3000973
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
  • Checked 165 log files, 37 edm output root files, 39 DQM output files
  • TriggerResults: no differences found

@smuzaffar
Copy link
Contributor

hold
this does not fix the ppc64le unit test failure issue. As this rebuild nearly all the externals so lets hold on to this update and we can include it either in newer archs e.g. gcc11 or we can merge it once we have to change any other base level package.

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 9, 2021

Pull request has been put on hold by @smuzaffar
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

@cmsbuild cmsbuild added the hold label Sep 9, 2021
@smuzaffar smuzaffar changed the base branch from IB/CMSSW_12_1_X/master to IB/CMSSW_12_2_X/master October 29, 2021 06:23
@smuzaffar smuzaffar changed the base branch from IB/CMSSW_12_2_X/master to IB/CMSSW_12_2_X/g11 November 4, 2021 15:36
@smuzaffar smuzaffar merged commit 08d766d into cms-sw:IB/CMSSW_12_2_X/g11 Nov 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants