[DEVEL] Tensorflow build recipe cleanup #5068

smuzaffar · 2019-07-03T09:24:15Z

fix eigen flags in toolfile: tensorflow is build with these extra flags
added extra pip packages dependencies for tensorflow
build py2-tensorflow using normal build-with-pip system to make sure that all of its deps are available
build tensorflow and protobuf with c++1z
combine tensoflow py2 and py2 build recipes
Make use of cms protobuf to build tensorflow

cmsbuild · 2019-07-03T09:24:43Z

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_11_0_X/gcc700next.

@cmsbuild, @smuzaffar, @gudrutis, @mrodozov can you please review it and eventually sign? Thanks.
cms-bot commands are listed here

smuzaffar · 2019-07-03T09:26:44Z

please test

cmsbuild · 2019-07-03T09:27:10Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1303/console Started: 2019/07/03 11:27

cmsbuild · 2019-07-03T10:31:18Z

-1

Tested at: 329ba87

Build:

I found compilation error when building:

build-external+tensorflow-sources+1.13.1-cms
* The action "install-external+python_tools+2.0-cms" was not completed successfully because The following dependencies could not complete:
build-external+python_tools+2.0-cms
* The action "build-external+tensorflow-sources+1.13.1-cms" was not completed successfully because Failed to build tensorflow-sources. Log file in /build/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/external/tensorflow-sources/1.13.1-cms/log. Final lines of the log file:
^
external/eigen_archive/Eigen/src/Core/products/GeneralBlockPanelKernel.h:1425:13: error: inconsistent operand constraints in an 'asm'
EIGEN_GEBGP_ONESTEP(6);
^
external/eigen_archive/Eigen/src/Core/products/GeneralBlockPanelKernel.h:1426:13: error: inconsistent operand constraints in an 'asm'
EIGEN_GEBGP_ONESTEP(7);
^

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-495834/1303/summary.html

smuzaffar · 2019-07-03T11:07:15Z

please test

cmsbuild · 2019-07-03T11:07:25Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/1306/console Started: 2019/07/03 13:08

cmsbuild · 2019-07-03T11:07:26Z

Pull request #5068 was updated.

cmsbuild · 2019-07-03T20:34:49Z

-1

Tested at: dd31d08

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-495834/1306/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests RelVals

Unit Tests:

I found errors in the following unit tests:

---> test materialBudgetTrackerPlots had ERRORS
---> test testTFMetaGraphLoading had ERRORS
---> test testTFGraphLoading had ERRORS
---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals:

The relvals timed out after 4 hours.
When I ran the RelVals I found an error in the following workflows:
136.8311 step2

runTheMatrix-results/136.8311_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017/step2_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017.log

136.7611 step2

runTheMatrix-results/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

136.788 step3

runTheMatrix-results/136.788_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017/step3_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017.log

135.4 step3

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step3_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

136.85 step3

runTheMatrix-results/136.85_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM/step3_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM.log

158.0 step3

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step3_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

10042.0 step3

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017/step3_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25.log

11624.0 step3

runTheMatrix-results/11624.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021.log

10824.0 step3

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

10224.0 step3

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU.log

20034.0 step3

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D17_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D17+RecoFullGlobal_2023D17+HARVESTFullGlobal_2023D17.log

21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D21_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D21+RecoFullGlobal_2023D21+HARVESTFullGlobal_2023D21/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D35+RecoFullGlobal_2023D35+HARVESTFullGlobal_2023D35.log

27434.0 step3

runTheMatrix-results/27434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D35+RecoFullGlobal_2023D35+HARVESTFullGlobal_2023D35/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D35+RecoFullGlobal_2023D35+HARVESTFullGlobal_2023D35.log

29034.0 step3

runTheMatrix-results/29034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D41+RecoFullGlobal_2023D41+HARVESTFullGlobal_2023D41/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2023D41+RecoFullGlobal_2023D41+HARVESTFullGlobal_2023D41.log

cmsbuild · 2019-07-03T20:34:52Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

smuzaffar added 2 commits July 3, 2019 11:19

cleanup tensorflow build recipe; build it with c++1z and cms protobuf

72dc96a

cleanup tensorflow build recipe; build it with c++1z and cms protobuf

329ba87

cmsbuild added comparison-pending externals-pending orp-pending pending-signatures tests-pending labels Jul 3, 2019

cmsbuild added tests-started and removed tests-pending labels Jul 3, 2019

cmsbuild added tests-rejected and removed tests-started labels Jul 3, 2019

take tensorflow for cms-externals

dd31d08

cmsbuild added tests-pending and removed tests-rejected labels Jul 3, 2019

cmsbuild added tests-started and removed tests-pending labels Jul 3, 2019

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Jul 3, 2019

smuzaffar merged commit 0705878 into IB/CMSSW_11_0_X/gcc700next Jul 3, 2019

smuzaffar deleted the tensorflow-113-cxx1z branch July 4, 2019 08:15

cmsbuild mentioned this pull request Jun 13, 2021

cleaning up unused py2 tools ; restoring tensorflow as part of python_tools.spec #7019

Merged

cmsbuild mentioned this pull request Jul 10, 2021

Update to python 3.9.6 #7111

Merged

cmsbuild mentioned this pull request Nov 21, 2022

SCRAM V2 disto via latest cmssw releases #8187

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DEVEL] Tensorflow build recipe cleanup #5068

[DEVEL] Tensorflow build recipe cleanup #5068

smuzaffar commented Jul 3, 2019

cmsbuild commented Jul 3, 2019

smuzaffar commented Jul 3, 2019

cmsbuild commented Jul 3, 2019 •

edited

Loading

cmsbuild commented Jul 3, 2019

smuzaffar commented Jul 3, 2019

cmsbuild commented Jul 3, 2019 •

edited

Loading

cmsbuild commented Jul 3, 2019

cmsbuild commented Jul 3, 2019

cmsbuild commented Jul 3, 2019

[DEVEL] Tensorflow build recipe cleanup #5068

[DEVEL] Tensorflow build recipe cleanup #5068

Conversation

smuzaffar commented Jul 3, 2019

cmsbuild commented Jul 3, 2019

smuzaffar commented Jul 3, 2019

cmsbuild commented Jul 3, 2019 • edited Loading

cmsbuild commented Jul 3, 2019

smuzaffar commented Jul 3, 2019

cmsbuild commented Jul 3, 2019 • edited Loading

cmsbuild commented Jul 3, 2019

cmsbuild commented Jul 3, 2019

cmsbuild commented Jul 3, 2019

cmsbuild commented Jul 3, 2019 •

edited

Loading

cmsbuild commented Jul 3, 2019 •

edited

Loading