Add a flag to prevent storage of LHEXMLStringProduct #22935
Conversation
…er which is no more working
|
The code-checks are being triggered in jenkins. |
|
@Dr15Jones @bendavid @franzoni @vlimant I assume that this is what we want to get rid of the memory problem in 93X (at least part of it) without losing the possibility to optionally save and extract the xml file |
|
-code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22935/4318 Code check has found code style and quality issues which could be resolved by applying a patch in https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22935/4318/git-diff.patch You can run |
|
The code-checks are being triggered in jenkins. |
From my point, it solves the file merge problem. It is unknown to me if this solves a memory problem as well. |
|
@Dr15Jones to be verified, I agree, but it is something that anyway we want to have |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-22935/4320 |
|
A new Pull Request was created by @fabiocos (Fabio Cossutti) for master. It involves the following packages: GeneratorInterface/LHEInterface @cmsbuild, @efeyazgan, @perrozzi can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
|
please test workflow 512 |
|
The tests are being triggered in jenkins. |
|
-1 Tested at: 1b9b2b8 You can see the results of the tests here: I found follow errors while testing this PR Failed tests: AddOn
I found errors in the following addon tests: cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_Fake2 --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2016 --fileout file:RelVal_Raw_Fake2_DATA.root --filein /store/data/Run2016B/JetHT/RAW/v1/000/272/762/00000/C666CDE2-E013-E611-B15A-02163E011DBE.root : FAILED - time: date Wed Apr 11 18:40:16 2018-date Wed Apr 11 18:35:14 2018 s - exit: 23552 |
|
Comparison job queued. |
|
The tests are being triggered in jenkins. |
|
Comparison job queued. |
|
Comparison is ready Comparison Summary:
|
|
@perrozzi @efeyazgan this change, previously discussed, is meant to be back-ported into 93X to verify to which extent it solves the problems found in production. Please check and sign or comment |
|
thanks Fabio. So, as far as I understand LHEXMLStringProduct was storing the LHE file as such and was used by the ExternalLHEAsciiDumper to dump it in case of need. |
|
@perrozzi in the present implementation nothing changes from the point of view of file structure (the product stays there) and provenance (I added an untracked parameter), but by default the product will be empty, so taking very little space. This should hopefully solve at least part of the problems keeping the production on hold (@kpedro88 FYI) The LHEXML product was meant to store the XML file as we were doing in the past, but I understand this is no more the default since quite sometime. The C++ translation is what the CMS FW is practically using in its workflow, the xml can be read to build it with the other code in the package, here it is done on the fly. I would suggest to move forward with this for the short term. If then the GEN group has plans for a deeper revision of the code he is welcome to present them. But be careful when you modify the file content, this should not happen in the middle of a production campaign. |
|
+1 |
cmsbuild
added
fully-signed
generators-approved
and removed
generators-pending
pending-signatures
labels
|
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2) |
|
+1 |
Backport of #22935 to 93X (Add a flag to prevent storage of LHEXMLStringProduct)
LHEXMLStringProduct is no more really used in production, and it is creating memory issues keeping on hold production in 93X. This PR adds a boolean flag as untracked parameter to activate/deactivate the storage of the xml output into the product, setting the default to "false" (i.e. do not store).
With the opportunity I fixed also the ExternalLHEAsciiDumper analyzer that was supposed to extract the xml file out of the edm root output, and that was no more working.
Both the "false" and "true" option have been tested in wf 512 (10 events), with a visible reduction of the final file size (84kB vs 112kB), and when the option was true the dumper was used to extract the xml file (216 kB after de-compression), proving that is again functional.