-- starting process: ['cmsRun', '/cmsnfsdqmdata/dqmdata/dqm_cmssw/production_0822_CMSSW_12_4_7/src/DQM/Integration/python/clients/beampixel_dqm_sourceclient-live_cfg.py', 'runInputDir=/fff/BU0/ramdisk', 'runNumber=357900', 'runkey=pp_run', 'runUniqueKey=40edef22-de13-40d4-9b27-db0f23155ccb'] --
Initial Source settings: cms.Source("DQMStreamerReader",
    SelectEvents = cms.untracked.vstring('*'),
    delayMillis = cms.untracked.uint32(500),
    deleteDatFiles = cms.untracked.bool(False),
    endOfRunKills = cms.untracked.bool(True),
    inputFileTransitionsEachEvent = cms.untracked.bool(False),
    minEventsPerLumi = cms.untracked.int32(1),
    nextLumiTimeoutMillis = cms.untracked.int32(240000),
    runInputDir = cms.untracked.string('/fff/BU0/ramdisk'),
    runNumber = cms.untracked.uint32(357900),
    scanOnce = cms.untracked.bool(False),
    skipFirstLumis = cms.untracked.bool(False),
    streamLabel = cms.untracked.string('streamDQM')
)

Loaded configuration file from: ['/etc/dqm_run_config']
dqmRunConfig: cms.PSet(
    collectorHost = cms.untracked.string('dqm-prod-local.cms'),
    collectorPort = cms.untracked.int32(9090),
    type = cms.untracked.string('production')
)
[beampixel_dqm_sourceclient-live_cfg]::running pp
[beampixel_dqm_sourceclient-live_cfg]::saving DIP file into cms.string('/nfshome0/dqmpro/BeamMonitorDQM/BeamPixelResults.txt')
Monitoring pipe: /tmp/dqm_monitoring/.es_monitoring_pid00008572
23-Aug-2022 10:40:00 CEST  Found and loaded json file: /fff/BU0/ramdisk/run357900/run357900_ls0002_streamDQM_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and skipped json file (stream label mismatch, streamDQMCalibration [files] != streamDQM [config]): /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQMCalibration_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and skipped json file (stream label mismatch, streamDQMGPUvsCPU [files] != streamDQM [config]): /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQMGPUvsCPU_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and skipped json file (stream label mismatch, streamDQMOnlineBeamspot [files] != streamDQM [config]): /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQMOnlineBeamspot_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and skipped json file (stream label mismatch, streamDQMEventDisplay [files] != streamDQM [config]): /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQMEventDisplay_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and skipped json file (stream label mismatch, streamDQMHistograms [files] != streamDQM [config]): /fff/BU0/ramdisk/run357900/run357900_ls0001_streamDQMHistograms_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and loaded json file: /fff/BU0/ramdisk/run357900/run357900_ls0001_streamDQM_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Waiting for the first lumi in order to initialize.
23-Aug-2022 10:40:00 CEST  Initiating request to open file /fff/BU0/ramdisk/run357900/run357900_ls0001_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:00 CEST  Successfully opened file /fff/BU0/ramdisk/run357900/run357900_ls0001_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:00 CEST  DQMStreamerReader initialised.
Begin processing the 1st record. Run 357900, Event 551089, LumiSection 1 on stream 0 at 23-Aug-2022 10:40:05.456 CEST
%MSG-w SiStripRawToDigi:  SiStripRawToDigiModule:siStripDigis  23-Aug-2022 10:40:23 CEST Run: 357900 Event: 551089
NULL pointer to FEDRawData for FED: id 353
Note: further warnings of this type will be suppressed (this can be changed by enabling debugging printout)
%MSG
PersistencyIO    INFO  +++ Set Streamer to dd4hep::OpaqueDataBlock
DD4hep           WARN  ++ Using globally Geant4 unit system (mm,ns,MeV)
DD4CMS           INFO  +++ Processing the CMS detector description xml-memory-buffer
Detector         INFO  *********** Created World volume with size: 101000 101000 450000
Detector         INFO  +++ Patching names of anonymous shapes....
DDDefinition     INFO  +++ Finished processing xml-memory-buffer
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 4.43099e-75
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 3.16918e-32
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 1.67629e-77
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 1.52265e-36
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 1.01836e-16
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 0
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 0
%MSG
23-Aug-2022 10:40:26 CEST  Found and loaded json file: /fff/BU0/ramdisk/run357900/run357900_ls0004_streamDQM_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:26 CEST  Found and loaded json file: /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQM_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:26 CEST  Closed file /fff/BU0/ramdisk/run357900/run357900_ls0001_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:26 CEST  Initiating request to open file /fff/BU0/ramdisk/run357900/run357900_ls0002_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:26 CEST  Successfully opened file /fff/BU0/ramdisk/run357900/run357900_ls0002_streamDQM_sm-c2a11-43-01.dat
Begin processing the 2nd record. Run 357900, Event 1814185, LumiSection 2 on stream 0 at 23-Aug-2022 10:40:26.992 CEST
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:27 CEST Run: 357900 Event: 1814185
rejected track t_tkwt 5.18949e-57
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:27 CEST Run: 357900 Event: 1814185
rejected track t_tkwt 2.03773e-18
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:27 CEST Run: 357900 Event: 1814185
rejected track t_tkwt 6.85333e-96
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:27 CEST Run: 357900 Event: 1814185
rejected track t_tkwt 3.01644e-30
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:27 CEST Run: 357900 Event: 1814185
rejected track t_tkwt 9.91654e-106
%MSG
23-Aug-2022 10:40:27 CEST  Closed file /fff/BU0/ramdisk/run357900/run357900_ls0002_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:27 CEST  Initiating request to open file /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:27 CEST  Successfully opened file /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQM_sm-c2a11-43-01.dat
%MSG-e Vx3DHLTAnalyzer:  Vx3DHLTAnalyzer:pixelVertexDQM@endLumi  23-Aug-2022 10:40:27 CEST Run: 357900 Lumi: 2
	Caught Minuit2 exception: An exception of category 'FatalRootError' occurred.
   Additional Info:
      [a] Fatal Root Error: @SUB=Minuit2
VariableMetricBuilder Initial matrix not pos.def.


%MSG
%MSG-e Vx3DHLTAnalyzer:  Vx3DHLTAnalyzer:pixelVertexDQM@endLumi  23-Aug-2022 10:40:27 CEST Run: 357900 Lumi: 2
	Caught Minuit2 exception: An exception of category 'FatalRootError' occurred.
   Additional Info:
      [a] Fatal Root Error: @SUB=Minuit2
VariableMetricBuilder Initial matrix not pos.def.


%MSG
%MSG-e Vx3DHLTAnalyzer:  Vx3DHLTAnalyzer:pixelVertexDQM@endLumi  23-Aug-2022 10:40:27 CEST Run: 357900 Lumi: 2
	Caught Minuit2 exception: An exception of category 'FatalRootError' occurred.
   Additional Info:
      [a] Fatal Root Error: @SUB=Minuit2
VariableMetricBuilder Initial matrix not pos.def.


%MSG


A fatal system signal has occurred: segmentation violation
The following is the call stack containing the origin of the signal.

Tue Aug 23 10:40:27 CEST 2022
Thread 3 (Thread 0x7f0f2487e700 (LWP 12842) "cmsRun"):
#0  0x00007f0f47a69b43 in select () from /lib64/libc.so.6
#1  0x00007f0f416afa92 in lat::IOSelectSelect::wait(long) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/external/slc7_amd64_gcc10/lib/libclasslib.so
#2  0x00007f0f416b055e in lat::IOSelector::wait(long) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/external/slc7_amd64_gcc10/lib/libclasslib.so
#3  0x00007f0f416b0318 in lat::IOSelector::dispatch(long) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/external/slc7_amd64_gcc10/lib/libclasslib.so
#4  0x00007f0f41719476 in DQMNet::run() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libDQMServicesCore.so
#5  0x00007f0f41719f0c in communicate(void*) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libDQMServicesCore.so
#6  0x00007f0f47d49ea5 in start_thread () from /lib64/libpthread.so.0
#7  0x00007f0f47a72b0d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7f0f25354700 (LWP 12673) "cmsRun"):
#0  0x00007f0f47d511d9 in waitpid () from /lib64/libpthread.so.0
#1  0x00007f0f41fb9487 in edm::service::cmssw_stacktrace_fork() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#2  0x00007f0f41fb9ffa in edm::service::InitRootHandlers::stacktraceHelperThread() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#3  0x00007f0f48348f90 in std::execute_native_thread_routine (__p=0x7f0f36e1b920) at ../../../../../libstdc++-v3/src/c++11/thread.cc:80
#4  0x00007f0f47d49ea5 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f0f47a72b0d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7f0f45bc5740 (LWP 12104) "cmsRun"):
#0  0x00007f0f47a67ddd in poll () from /lib64/libc.so.6
#1  0x00007f0f41fb973f in full_read.constprop () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#2  0x00007f0f41fba0cc in edm::service::InitRootHandlers::stacktraceFromThread() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#3  0x00007f0f41fbca1b in sig_dostack_then_abort () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  0x00007f0eff57d793 in Vx3DHLTAnalyzer::MyFit(std::vector<double, std::allocator<double> >*) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginVx3DHLTAnalyzer.so
#6  0x00007f0eff583e37 in Vx3DHLTAnalyzer::dqmEndLuminosityBlock(edm::LuminosityBlock const&, edm::EventSetup const&) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginVx3DHLTAnalyzer.so
#7  0x00007f0eff590885 in non-virtual thunk to DQMOneLumiEDAnalyzer<>::endLuminosityBlockProduce(edm::LuminosityBlock&, edm::EventSetup const&) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginVx3DHLTAnalyzer.so
#8  0x00007f0f4a4eb6dd in edm::one::EDProducerBase::doEndLuminosityBlock(edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#9  0x00007f0f4a4d0bd0 in edm::WorkerT<edm::one::EDProducerBase>::implDoEnd(edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#10 0x00007f0f4a3d6d64 in decltype ({parm#1}()) edm::convertException::wrap<edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::Context const*)::{lambda()#1}>(edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::Context const*)::{lambda()#1}) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#11 0x00007f0f4a3d7117 in std::__exception_ptr::exception_ptr edm::Worker::runModuleAfterAsyncPrefetch<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >(std::__exception_ptr::exception_ptr const*, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::Context const*) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#12 0x00007f0f4a3d76a8 in void edm::SerialTaskQueueChain::actionToRun<edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >::execute()::{lambda()#1}&>(edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >::execute()::{lambda()#1}&) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#13 0x00007f0f4a3d7791 in edm::SerialTaskQueue::QueuedTask<edm::SerialTaskQueueChain::push<edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >::execute()::{lambda()#1}&>(tbb::detail::d1::task_group&, edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >::execute()::{lambda()#1}&)::{lambda()#1}>::execute() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#14 0x00007f0f4a62f055 in tbb::detail::d1::function_task<edm::SerialTaskQueue::spawn(edm::SerialTaskQueue::TaskBase&)::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreConcurrency.so
#15 0x00007f0f48b4ebec in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter> (waiter=..., t=0x7f0f451b2700, this=0x7f0f451df700) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/build/CMSSW_12_4_0_pre2-build/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-e966a5acb1e4d5fd7605074bafbb079c/tbb-v2021.5.0/src/tbb/task_dispatcher.h:322
#16 tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter> (waiter=..., t=<optimized out>, this=0x7f0f451df700) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/build/CMSSW_12_4_0_pre2-build/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-e966a5acb1e4d5fd7605074bafbb079c/tbb-v2021.5.0/src/tbb/task_dispatcher.h:463
#17 tbb::detail::r1::task_dispatcher::execute_and_wait (t=<optimized out>, wait_ctx=..., w_ctx=...) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/build/CMSSW_12_4_0_pre2-build/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-e966a5acb1e4d5fd7605074bafbb079c/tbb-v2021.5.0/src/tbb/task_dispatcher.cpp:168
#18 0x00007f0f4a3a0368 in edm::EventProcessor::processLumis(std::shared_ptr<void> const&) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#19 0x00007f0f4a3ab28b in edm::EventProcessor::runToCompletion() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#20 0x000000000040a266 in tbb::detail::d1::task_arena_function<main::{lambda()#1}::operator()() const::{lambda()#1}, void>::operator()() const ()
#21 0x00007f0f48b3d0eb in tbb::detail::r1::task_arena_impl::execute (ta=..., d=...) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/build/CMSSW_12_4_0_pre2-build/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-e966a5acb1e4d5fd7605074bafbb079c/tbb-v2021.5.0/src/tbb/arena.cpp:698
 ... 
Initial Source settings: cms.Source("DQMStreamerReader",
    SelectEvents = cms.untracked.vstring('*'),
    delayMillis = cms.untracked.uint32(500),
    deleteDatFiles = cms.untracked.bool(False),
    endOfRunKills = cms.untracked.bool(True),
    inputFileTransitionsEachEvent = cms.untracked.bool(False),
    minEventsPerLumi = cms.untracked.int32(1),
    nextLumiTimeoutMillis = cms.untracked.int32(240000),
    runInputDir = cms.untracked.string('/fff/BU0/ramdisk'),
    runNumber = cms.untracked.uint32(357900),
    scanOnce = cms.untracked.bool(False),
    skipFirstLumis = cms.untracked.bool(False),
    streamLabel = cms.untracked.string('streamDQM')
)

Loaded configuration file from: ['/etc/dqm_run_config']
dqmRunConfig: cms.PSet(
    collectorHost = cms.untracked.string('dqm-prod-local.cms'),
    collectorPort = cms.untracked.int32(9090),
    type = cms.untracked.string('production')
)
[beampixel_dqm_sourceclient-live_cfg]::running pp
[beampixel_dqm_sourceclient-live_cfg]::saving DIP file into cms.string('/nfshome0/dqmpro/BeamMonitorDQM/BeamPixelResults.txt')
Monitoring pipe: /tmp/dqm_monitoring/.es_monitoring_pid00008572
23-Aug-2022 10:40:00 CEST  Found and loaded json file: /fff/BU0/ramdisk/run357900/run357900_ls0002_streamDQM_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and skipped json file (stream label mismatch, streamDQMCalibration [files] != streamDQM [config]): /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQMCalibration_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and skipped json file (stream label mismatch, streamDQMGPUvsCPU [files] != streamDQM [config]): /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQMGPUvsCPU_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and skipped json file (stream label mismatch, streamDQMOnlineBeamspot [files] != streamDQM [config]): /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQMOnlineBeamspot_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and skipped json file (stream label mismatch, streamDQMEventDisplay [files] != streamDQM [config]): /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQMEventDisplay_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and skipped json file (stream label mismatch, streamDQMHistograms [files] != streamDQM [config]): /fff/BU0/ramdisk/run357900/run357900_ls0001_streamDQMHistograms_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Found and loaded json file: /fff/BU0/ramdisk/run357900/run357900_ls0001_streamDQM_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:00 CEST  Waiting for the first lumi in order to initialize.
23-Aug-2022 10:40:00 CEST  Initiating request to open file /fff/BU0/ramdisk/run357900/run357900_ls0001_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:00 CEST  Successfully opened file /fff/BU0/ramdisk/run357900/run357900_ls0001_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:00 CEST  DQMStreamerReader initialised.
Begin processing the 1st record. Run 357900, Event 551089, LumiSection 1 on stream 0 at 23-Aug-2022 10:40:05.456 CEST
%MSG-w SiStripRawToDigi:  SiStripRawToDigiModule:siStripDigis  23-Aug-2022 10:40:23 CEST Run: 357900 Event: 551089
NULL pointer to FEDRawData for FED: id 353
Note: further warnings of this type will be suppressed (this can be changed by enabling debugging printout)
%MSG
PersistencyIO    INFO  +++ Set Streamer to dd4hep::OpaqueDataBlock
DD4hep           WARN  ++ Using globally Geant4 unit system (mm,ns,MeV)
DD4CMS           INFO  +++ Processing the CMS detector description xml-memory-buffer
Detector         INFO  *********** Created World volume with size: 101000 101000 450000
Detector         INFO  +++ Patching names of anonymous shapes....
DDDefinition     INFO  +++ Finished processing xml-memory-buffer
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 4.43099e-75
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 3.16918e-32
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 1.67629e-77
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 1.52265e-36
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 1.01836e-16
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 0
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:26 CEST Run: 357900 Event: 551089
rejected track t_tkwt 0
%MSG
23-Aug-2022 10:40:26 CEST  Found and loaded json file: /fff/BU0/ramdisk/run357900/run357900_ls0004_streamDQM_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:26 CEST  Found and loaded json file: /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQM_sm-c2a11-43-01.jsn
23-Aug-2022 10:40:26 CEST  Closed file /fff/BU0/ramdisk/run357900/run357900_ls0001_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:26 CEST  Initiating request to open file /fff/BU0/ramdisk/run357900/run357900_ls0002_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:26 CEST  Successfully opened file /fff/BU0/ramdisk/run357900/run357900_ls0002_streamDQM_sm-c2a11-43-01.dat
Begin processing the 2nd record. Run 357900, Event 1814185, LumiSection 2 on stream 0 at 23-Aug-2022 10:40:26.992 CEST
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:27 CEST Run: 357900 Event: 1814185
rejected track t_tkwt 5.18949e-57
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:27 CEST Run: 357900 Event: 1814185
rejected track t_tkwt 2.03773e-18
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:27 CEST Run: 357900 Event: 1814185
rejected track t_tkwt 6.85333e-96
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:27 CEST Run: 357900 Event: 1814185
rejected track t_tkwt 3.01644e-30
%MSG
%MSG-w DAClusterizerinZ_vect:  PrimaryVertexProducer:pixelVertices  23-Aug-2022 10:40:27 CEST Run: 357900 Event: 1814185
rejected track t_tkwt 9.91654e-106
%MSG
23-Aug-2022 10:40:27 CEST  Closed file /fff/BU0/ramdisk/run357900/run357900_ls0002_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:27 CEST  Initiating request to open file /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQM_sm-c2a11-43-01.dat
23-Aug-2022 10:40:27 CEST  Successfully opened file /fff/BU0/ramdisk/run357900/run357900_ls0003_streamDQM_sm-c2a11-43-01.dat
%MSG-e Vx3DHLTAnalyzer:  Vx3DHLTAnalyzer:pixelVertexDQM@endLumi  23-Aug-2022 10:40:27 CEST Run: 357900 Lumi: 2
	Caught Minuit2 exception: An exception of category 'FatalRootError' occurred.
   Additional Info:
      [a] Fatal Root Error: @SUB=Minuit2
VariableMetricBuilder Initial matrix not pos.def.


%MSG
%MSG-e Vx3DHLTAnalyzer:  Vx3DHLTAnalyzer:pixelVertexDQM@endLumi  23-Aug-2022 10:40:27 CEST Run: 357900 Lumi: 2
	Caught Minuit2 exception: An exception of category 'FatalRootError' occurred.
   Additional Info:
      [a] Fatal Root Error: @SUB=Minuit2
VariableMetricBuilder Initial matrix not pos.def.


%MSG
%MSG-e Vx3DHLTAnalyzer:  Vx3DHLTAnalyzer:pixelVertexDQM@endLumi  23-Aug-2022 10:40:27 CEST Run: 357900 Lumi: 2
	Caught Minuit2 exception: An exception of category 'FatalRootError' occurred.
   Additional Info:
      [a] Fatal Root Error: @SUB=Minuit2
VariableMetricBuilder Initial matrix not pos.def.


%MSG


A fatal system signal has occurred: segmentation violation
The following is the call stack containing the origin of the signal.

Tue Aug 23 10:40:27 CEST 2022
Thread 3 (Thread 0x7f0f2487e700 (LWP 12842) "cmsRun"):
#0  0x00007f0f47a69b43 in select () from /lib64/libc.so.6
#1  0x00007f0f416afa92 in lat::IOSelectSelect::wait(long) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/external/slc7_amd64_gcc10/lib/libclasslib.so
#2  0x00007f0f416b055e in lat::IOSelector::wait(long) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/external/slc7_amd64_gcc10/lib/libclasslib.so
#3  0x00007f0f416b0318 in lat::IOSelector::dispatch(long) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/external/slc7_amd64_gcc10/lib/libclasslib.so
#4  0x00007f0f41719476 in DQMNet::run() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libDQMServicesCore.so
#5  0x00007f0f41719f0c in communicate(void*) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libDQMServicesCore.so
#6  0x00007f0f47d49ea5 in start_thread () from /lib64/libpthread.so.0
#7  0x00007f0f47a72b0d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7f0f25354700 (LWP 12673) "cmsRun"):
#0  0x00007f0f47d511d9 in waitpid () from /lib64/libpthread.so.0
#1  0x00007f0f41fb9487 in edm::service::cmssw_stacktrace_fork() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#2  0x00007f0f41fb9ffa in edm::service::InitRootHandlers::stacktraceHelperThread() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#3  0x00007f0f48348f90 in std::execute_native_thread_routine (__p=0x7f0f36e1b920) at ../../../../../libstdc++-v3/src/c++11/thread.cc:80
#4  0x00007f0f47d49ea5 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f0f47a72b0d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7f0f45bc5740 (LWP 12104) "cmsRun"):
#0  0x00007f0f47a67ddd in poll () from /lib64/libc.so.6
#1  0x00007f0f41fb973f in full_read.constprop () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#2  0x00007f0f41fba0cc in edm::service::InitRootHandlers::stacktraceFromThread() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#3  0x00007f0f41fbca1b in sig_dostack_then_abort () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  0x00007f0eff57d793 in Vx3DHLTAnalyzer::MyFit(std::vector<double, std::allocator<double> >*) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginVx3DHLTAnalyzer.so
#6  0x00007f0eff583e37 in Vx3DHLTAnalyzer::dqmEndLuminosityBlock(edm::LuminosityBlock const&, edm::EventSetup const&) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginVx3DHLTAnalyzer.so
#7  0x00007f0eff590885 in non-virtual thunk to DQMOneLumiEDAnalyzer<>::endLuminosityBlockProduce(edm::LuminosityBlock&, edm::EventSetup const&) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/pluginVx3DHLTAnalyzer.so
#8  0x00007f0f4a4eb6dd in edm::one::EDProducerBase::doEndLuminosityBlock(edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#9  0x00007f0f4a4d0bd0 in edm::WorkerT<edm::one::EDProducerBase>::implDoEnd(edm::LumiTransitionInfo const&, edm::ModuleCallingContext const*) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#10 0x00007f0f4a3d6d64 in decltype ({parm#1}()) edm::convertException::wrap<edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::Context const*)::{lambda()#1}>(edm::Worker::runModule<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >(edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::Context const*)::{lambda()#1}) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#11 0x00007f0f4a3d7117 in std::__exception_ptr::exception_ptr edm::Worker::runModuleAfterAsyncPrefetch<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >(std::__exception_ptr::exception_ptr const*, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3>::Context const*) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#12 0x00007f0f4a3d76a8 in void edm::SerialTaskQueueChain::actionToRun<edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >::execute()::{lambda()#1}&>(edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >::execute()::{lambda()#1}&) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#13 0x00007f0f4a3d7791 in edm::SerialTaskQueue::QueuedTask<edm::SerialTaskQueueChain::push<edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >::execute()::{lambda()#1}&>(tbb::detail::d1::task_group&, edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::LuminosityBlockPrincipal, (edm::BranchActionType)3> >::execute()::{lambda()#1}&)::{lambda()#1}>::execute() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#14 0x00007f0f4a62f055 in tbb::detail::d1::function_task<edm::SerialTaskQueue::spawn(edm::SerialTaskQueue::TaskBase&)::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreConcurrency.so
#15 0x00007f0f48b4ebec in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter> (waiter=..., t=0x7f0f451b2700, this=0x7f0f451df700) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/build/CMSSW_12_4_0_pre2-build/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-e966a5acb1e4d5fd7605074bafbb079c/tbb-v2021.5.0/src/tbb/task_dispatcher.h:322
#16 tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter> (waiter=..., t=<optimized out>, this=0x7f0f451df700) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/build/CMSSW_12_4_0_pre2-build/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-e966a5acb1e4d5fd7605074bafbb079c/tbb-v2021.5.0/src/tbb/task_dispatcher.h:463
#17 tbb::detail::r1::task_dispatcher::execute_and_wait (t=<optimized out>, wait_ctx=..., w_ctx=...) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/build/CMSSW_12_4_0_pre2-build/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-e966a5acb1e4d5fd7605074bafbb079c/tbb-v2021.5.0/src/tbb/task_dispatcher.cpp:168
#18 0x00007f0f4a3a0368 in edm::EventProcessor::processLumis(std::shared_ptr<void> const&) () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#19 0x00007f0f4a3ab28b in edm::EventProcessor::runToCompletion() () from /opt/offline/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_7/lib/slc7_amd64_gcc10/libFWCoreFramework.so
#20 0x000000000040a266 in tbb::detail::d1::task_arena_function<main::{lambda()#1}::operator()() const::{lambda()#1}, void>::operator()() const ()
#21 0x00007f0f48b3d0eb in tbb::detail::r1::task_arena_impl::execute (ta=..., d=...) at /data/cmsbld/jenkins/workspace/auto-builds/CMSSW_12_4_0_pre2-slc7_amd64_gcc10/build/CMSSW_12_4_0_pre2-build/BUILD/slc7_amd64_gcc10/external/tbb/v2021.5.0-e966a5acb1e4d5fd7605074bafbb079c/tbb-v2021.5.0/src/tbb/arena.cpp:698
#22 0x000000000040b094 in main::{lambda()#1}::operator()() const ()
#23 0x000000000040971c in main ()

Current Modules:

Module: Vx3DHLTAnalyzer:pixelVertexDQM (crashed)

A fatal system signal has occurred: segmentation violation

-- process exit: -11 --