Failed to build dqmgui

[ywkao]

Dear cms-DQM experts,

Following the instructions on README.md [1], I could not build a dqmgui on a CC7 virtual machine [2]. When executing the command [3] in a clean environment (i.e., without cmsenv), error messages will appear, as shown in this log file [4] and on the screen [5]. Are there any suggestions on how to fix the problem?

Best regards,
Yu-Wei

[1] https://github.com/cms-DQM/dqmgui_prod/blob/index128/README.md
In cmsdist/dqmgui.spec, one line is modified as follows:

# Source0: git+https://github.com/cms-DQM/dqmgui_prod.git?obj=index128/%realversion&export=Monitoring&output=/Monitoring.tar.gz
Source0: git+https://github.com/cms-DQM/dqmgui_prod.git?obj=dev128/%realversion&export=Monitoring&output=/Monitoring.tar.gz

[2] A screenshot of a virtual machine on OpenStack service
https://ykao.web.cern.ch/ykao/raw_data_handling/hgcal_dqm_gui/screenshot_virtual_machine_cc7_openStackService.png

The virtual machine is built following the two instructions:
https://clouddocs.web.cern.ch/tutorial/index.html
https://twiki.cern.ch/twiki/bin/viewauth/CMS/DQMGuiForUsers#How

[3] Executed command

./pkgtools/cmsBuild --repo comp -a slc7_amd64_gcc630 -i w -j 10 build dqmgui

[4] Log messages
https://ykao.web.cern.ch/ykao/raw_data_handling/hgcal_dqm_gui/log_dqmgui_prod.txt

[5] Error messages on screen

Done fetching /mnt/work/dev_DQM_GUI_for_help/cmsdist/dqmgui-rtgu.patch
(<function fetchSources at 0x7f614b071ed8>, <Package name=dqmgui>, <scheduler.Scheduler object at 0x7f6145fb1810>) done
Creating directory /mnt/work/dev_DQM_GUI_for_help/wprime/BUILD/slc7_amd64_gcc630/cms/dqmgui/9.7.8-8f294bf9497ba9a6fcf6030e067117e7 if not existing.
Building dqmgui. Log can be found in /mnt/work/dev_DQM_GUI_for_help/wprime/BUILD/slc7_amd64_gcc630/cms/dqmgui/9.7.8-8f294bf9497ba9a6fcf6030e067117e7/log.
rm -rf /mnt/work/dev_DQM_GUI_for_help/wprime/BUILD/slc7_amd64_gcc630/cms/dqmgui/9.7.8-8f294bf9497ba9a6fcf6030e067117e7/pkgtools-pkg-src-move2git /mnt/work/dev_DQM_GUI_for_help/wprime/BUILD/slc7_amd64_gcc630/cms/dqmgui/9.7.8-8f294bf9497ba9a6fcf6030e067117e7/cmsdist-tmp; export _CMSBUILD_BUILD_ENV_=1; mkdir -p /mnt/work/dev_DQM_GUI_for_help/wprime/BUILD/slc7_amd64_gcc630/cms/dqmgui/9.7.8-8f294bf9497ba9a6fcf6030e067117e7/cmsdist-tmp; source /mnt/work/dev_DQM_GUI_for_help/wprime/slc7_amd64_gcc630/rpm-env.sh ;TMPDIR=/mnt/work/dev_DQM_GUI_for_help/wprime/BUILD/slc7_amd64_gcc630/cms/dqmgui/9.7.8-8f294bf9497ba9a6fcf6030e067117e7/cmsdist-tmp  rpmbuild  --buildroot /mnt/work/dev_DQM_GUI_for_help/wprime/tmp/BUILDROOT/8f294bf9497ba9a6fcf6030e067117e7 -ba --define '_topdir /mnt/work/dev_DQM_GUI_for_help/wprime' --define "compiling_processes 2"  --define "cmsdist_directory /mnt/work/dev_DQM_GUI_for_help/cmsdist" --define "compilerv 630" --define "cmscompilerv 630" --define "cmsos slc7_amd64" --define "package_vectorization %{nil}" /mnt/work/dev_DQM_GUI_for_help/wprime/SPECS/cms/dqmgui/9.7.8-8f294bf9497ba9a6fcf6030e067117e7/spec >/mnt/work/dev_DQM_GUI_for_help/wprime/BUILD/slc7_amd64_gcc630/cms/dqmgui/9.7.8-8f294bf9497ba9a6fcf6030e067117e7/log 2>&1
(<function buildPackage at 0x7f614b072398>, <Package name=dqmgui>, <scheduler.Scheduler object at 0x7f6145fb1810>) done
Requested to quit.
Requested to quit.
Requested to quit.
Requested to quit.
* The action "install-cms+dqmgui+9.7.8-8f294bf9497ba9a6fcf6030e067117e7" was not completed successfully because The following dependencies could not complete:
build-cms+dqmgui+9.7.8-8f294bf9497ba9a6fcf6030e067117e7
* The action "install-cms+dqmgui-webdoc+9.7.8-8f294bf9497ba9a6fcf6030e067117e7" was not completed successfully because The following dependencies could not complete:
install-cms+dqmgui+9.7.8-8f294bf9497ba9a6fcf6030e067117e7
* The action "build-cms+dqmgui+9.7.8-8f294bf9497ba9a6fcf6030e067117e7" was not completed successfully because Failed to build dqmgui. Log file in /mnt/work/dev_DQM_GUI_for_help/wprime/BUILD/slc7_amd64_gcc630/cms/dqmgui/9.7.8-8f294bf9497ba9a6fcf6030e067117e7/log. Final lines of the log file:
./DQM/StringAtom.h:59:17: warning: left shift of negative value [-Wshift-negative-value]
& ~(~0 << n)));
^
./DQM/serverext.cc: In function 'std::__cxx11::string stringsToJSON(const StringAtomSet&, bool)':
./DQM/serverext.cc:536:53: warning: unused parameter 'emptyIsNone' [-Wunused-parameter]
bool emptyIsNone = false) {
^~~~~
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.
make: *** [serverext.o] Error 4
make: Target `all' not remade because of errors.
error: command 'make' failed with exit status 2
error: Bad exit status from /mnt/work/dev_DQM_GUI_for_help/wprime/tmp/rpm-tmp.m7zPed (%build)


RPM build errors:
Macro %rpmbuild_libdir defined but not used within scope
Bad exit status from /mnt/work/dev_DQM_GUI_for_help/wprime/tmp/rpm-tmp.m7zPed (%build)

* The action "final-job" was not completed successfully because The following dependencies could not complete:
build-cms+dqmgui+9.7.8-8f294bf9497ba9a6fcf6030e067117e7

[jfernan2]

Hi @ywkao
IIUC, if your intention is to set up your own DQM GUI for development, you don't need to build the code as your reference [1] reads (which are intructions to create a rpm indeed), but simply follow instead:

https://twiki.cern.ch/twiki/bin/viewauth/CMS/DQMGuiForUsers#How_to_set_up_your_own_DQM_GUI_f

and make sure in your OpenStack VM that the following packages are installed, by doing: (not needed on lxplus)
For CC7: sudo yum -y install git bzip2 perl-Switch perl-Env perl-Thread-Queue libXpm-devel libXmu libXext-devel mesa-libGLU-devel libXinerama libXi libXft-devel libXrandr libXcursor zsh tk perl-ExtUtils-Embed compat-libstdc++-33

I have just tried on lxplus with the latest deployment stable version (HG2305a) and it worked out of the box.
Best

[ywkao]

Hi @jfernan2,

Thank you for your response. Sorry for not making it clear first time. We would like to add a new feature in the dqm gui. That is why we need to build from this repository. We have successfully built a DQM GUI following the twiki page before.

The new feature we want to add is to display TH2Poly histograms. We implement the idea in DQM services in a user branch of cmssw [1] and store a TH2Poly object in a DQM root file [2][3].

However, the hexagonal histogram fails to display on DQM GUI [4]. We noticed that the issue of the new feature might be related to the code here because TH2Poly needs to be recognizable. Similar to [1], a new type of MonitorElement is implemented in a user branch [5]. It will be great if there is a way to test the branch.

Would you have a recommendation on how to best proceed on this front?

Best regards,
Yu-Wei

[1] ywkao/cmssw@d9e70fc

[2] A screenshot of a hexagonal histogram with pedestal values
https://ykao.web.cern.ch/ykao/raw_data_handling/output_DQMEDAnalyzer_cmssw_13_1_X/screenshot_DQM_hexagonal_histogram.png

[3] A DQM root file: /afs/cern.ch/user/y/ykao/work/public/example_HGCAL_DQM/DQM_V0001_HGCAL_R000123465.root

[4] https://ykao.web.cern.ch/ykao/raw_data_handling/output_DQMEDAnalyzer_cmssw_13_1_X/screenshot_hex_plots_not_shown.png

[5] ywkao@09a4a17

[jfernan2]

Thanks @ywkao , now I understand.

The commands listed in the README work fine for me in lxplus, so I believe this is an issue of OpenStack/VM and the way it compiles, indeed my (successful) log is identical to yours (with the same warnings) up to your lines:

< c++: internal compiler error: Killed (program cc1plus)
< Please submit a full bug report,
< with preprocessed source if appropriate.
< See http://gcc.gnu.org/bugs.html for instructions.
< make: *** [serverext.o] Error 4
< make: Target `all' not remade because of errors.
< error: command 'make' failed with exit status 2
< error: Bad exit status from /mnt/work/dev_DQM_GUI_for_help/w/tmp/rpm-tmp.ikBWVh (%build)

and then in my case the build goes on...until the end.

Since the error message you quote is a bit cryptic or meaningless at least, could you try to rerun again with --debug option to see if we get something else?

./pkgtools/cmsBuild --repo comp -a slc7_amd64_gcc630 -i x -j 10 build dqmgui --debug

[ywkao]

Hi @jfernan2,

Thank you for the information. Log messages on the virtual machine with the debug option can be found here [1][2]. The contents are almost the same as the previous ones.

I give it a try on lxplus7, and the command works for me as well! The corresponding log file can be found here [3]. Then, I move to the next step, following the README.md. Yet, I do not see any rpm files under the directory, w/BUILD/slc7_amd64_gcc630/cms/dqmgui/9.7.8-420be9fab47b3df007de6edc882eb950/. Instead, I found three rpm files under the directories of SRPMS and RPMS:

/afs/cern.ch/work/y/ykao/pure_test/w/SRPMS/cache/420be9fab47b3df007de6edc882eb950/cms+dqmgui+9.7.8-420be9fab47b3df007de6edc882eb950-1-1.src.rpm
/afs/cern.ch/work/y/ykao/pure_test/w/RPMS/cache/420be9fab47b3df007de6edc882eb950/slc7_amd64_gcc630/cms+dqmgui+9.7.8-420be9fab47b3df007de6edc882eb950-1-1.slc7_amd64_gcc630.rpm
/afs/cern.ch/work/y/ykao/pure_test/w/RPMS/cache/420be9fab47b3df007de6edc882eb950/slc7_amd64_gcc630/cms+dqmgui-webdoc+9.7.8-420be9fab47b3df007de6edc882eb950-1-1.slc7_amd64_gcc630.rpm

A few questions I have:

1. Which rpm file should I use?
2. How should I test/compile the branch with new features [4]?
3. Can I use the produced rpm file in the testGUI built from deployment [5]? Like replace the directory, current/sw/slc7_amd64_gcc630/cms/dqmgui/9.7.8, with the one extracted from an rpm file?

[1] On the virtual machine, the log file in x/BUILD/slc7_amd64_gcc630/cms/dqmgui:
https://ykao.web.cern.ch/ykao/raw_data_handling/hgcal_dqm_gui/log_dqmgui_prod_debug_20230525.txt
[2] On the virtual machine, the log messages displayed on the screen https://ykao.web.cern.ch/ykao/raw_data_handling/hgcal_dqm_gui/log_dqmgui_prod_debug_onScreen_20230525.txt
[3] On lxplus7, the log file in w/BUILD/slc7_amd64_gcc630/cms/dqmgui
https://ykao.web.cern.ch/ykao/raw_data_handling/hgcal_dqm_gui/log_dqmgui_prod_lxplus.txt
[4] ywkao@faf05ce
[5] https://twiki.cern.ch/twiki/bin/viewauth/CMS/DQMGuiForUsers#How

[jfernan2]

Hi @ywkao
you should:

1. Use cms+dqmgui+9.7.8-420be9fab47b3df007de6edc882eb950-1-1.slc7_amd64_gcc630.rpm

2. Create a new rpm with your modifed version of dqmgui code in dqmgui.spec from :
   Source0: git+https://github.com/ywkao/dqmgui_prod.git?obj=WHATEVER/%realversion&export=Monitoring&output=/Monitoring.tar.gz

being WHATEVER = th2poly ???

3. Go to your testGUI built form deployment and replace current/sw/slc7_amd64_gcc630/cms/dqmgui/9.7.8 with what you extract from:

rpm2cpio myrpmfile.rpm | cpio -idmv

being myrpmfile.rpm the new rpm built with you modified dqmgui code

HTH

[ywkao]

Hi @jfernan2,
Thank you so much for the guidelines! Finally, the monitor elements of TH2Poly can be displayed! The link here shows a screenshot [1]. The detailed steps we did are put as a record here [2].

We are considering testing with a few more wafer maps on the DQM GUI. If we want to implement the feature in the cmssw, shall we create two PRs? One on this repository and the other on the cmssw?

Best regards,
Yu-Wei

[1] https://ykao.web.cern.ch/ykao/raw_data_handling/hgcal_dqm_gui/screenshot_th2poly_successfully_displayed.png
[2] A note on the procedure tested

• Step1: Create a tag based on a branch, like this one https://github.com/ywkao/dqmgui_prod/releases/tag/9.7.8.2-alpha.4
• Step2: Modify cmsdist/dqmgui.spec (tag name in the first line and branch name in the source0)

### RPM cms dqmgui 9.7.8.2-alpha.4
Source0: git+https://github.com/ywkao/dqmgui_prod.git?obj=th2poly/%realversion&export=Monitoring&output=/Monitoring.tar.gz

• Step3: Build another testGui from scratch following the twiki page, https://twiki.cern.ch/twiki/bin/viewauth/CMS/DQMGuiForUsers#How
• Step4: Extract files from an rpm file
• Step5: Replace files under the bin, include, and shared libraries under current/sw/slc7_amd64_gcc630/cms/dqmgui/9.7.8/128.

# A list of replaced files are shown as follows
include/*
bin/DQMCollector
bin/visDQMIndex
bin/visDQMRender
lib/python2.7/site-packages/Monitoring/DQM/Accelerator.so
lib/libDQMGUI.so

• Step6: source env.sh and launch a DQM GUI

source current/apps/dqmgui/128/etc/profile.d/env.sh
$PWD/current/config/dqmgui/manage -f dev start "I did read documentation"

[jfernan2]

Hi @ywkao
Congratulations.

Yes, you need two PRs, one in cmssw to introduce THPoly in DQM production root files and the other PR in dmqgui_prod itself.

Please be sure to link in the comments both PRs somehow with a reference to this issue for example, so that the @cms-sw/dqm-l2 managers can be aware of this test and procedure.

Thanks

[pmandrik]

./DQM/serverext.cc:536:53: warning: unused parameter 'emptyIsNone' [-Wunused-parameter]
bool emptyIsNone = false) {
^~~~~
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if

Can you be out of memory at your machine? You may try to run using one thread with -j 1 option.

[nothingface0]

@ywkao and anyone interested, I know it's been a long time, but you can always follow the (kind of) updated instructions here (for CC7):

https://docs.google.com/document/d/1ue7wWEcBJUf-JCJ3qtDVwlWAZ9-iTOm7TnoFYxPjCKU/edit

We tested this procedure recently.