Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

L1TStage2EMTF seg faulting with change to DQMEDAnalyzer #29096

Open
Dr15Jones opened this issue Mar 4, 2020 · 28 comments
Open

L1TStage2EMTF seg faulting with change to DQMEDAnalyzer #29096

Dr15Jones opened this issue Mar 4, 2020 · 28 comments

Comments

@Dr15Jones
Copy link
Contributor

It IB is showing L1TStage2EMTF causing segmentation faults after the change to stream module.

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 4, 2020

A new Issue was created by @Dr15Jones Chris Jones.

@Dr15Jones, @smuzaffar, @silviodonato, @makortel, @davidlange6, @fabiocos can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

@Dr15Jones
Copy link
Contributor Author

The traceback is

#3  0x00002b077dfd1a99 in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/nweek-02618/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-01-2300/lib/slc7_amd64_gcc820/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  0x00002b07731b5aa9 in __memcpy_ssse3_back () from /lib64/libc.so.6
#6  0x00002b0771761209 in TArrayD::Set(int) [clone .localalias.7] () from /cvmfs/cms-ib.cern.ch/nweek-02618/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_1_X_2020-03-03-2300/external/slc7_amd64_gcc820/lib/libCore.so
#7  0x00002b0788d6b2cf in TH1::Sumw2(bool) () from /cvmfs/cms-ib.cern.ch/nweek-02618/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_1_X_2020-03-03-2300/external/slc7_amd64_gcc820/lib/libHist.so
#8  0x00002b0788d756ca in TH1::Divide(TH1 const*, TH1 const*, double, double, char const*) () from /cvmfs/cms-ib.cern.ch/nweek-02618/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_1_X_2020-03-03-2300/external/slc7_amd64_gcc820/lib/libHist.so
#9  0x00002b07db8b7c9c in L1TStage2EMTF::analyze(edm::Event const&amp, edm::EventSetup const&amp) () from /cvmfs/cms-ib.cern.ch/nweek-02618/slc7_amd64_gcc820/cms/cmssw-patch/CMSSW_11_1_X_2020-03-03-2300/lib/slc7_amd64_gcc820/libDQML1TMonitor.so
#10 0x00002b0770944dd6 in edm::stream::EDProducerAdaptorBase::doEvent(edm::EventPrincipal const&amp, edm::EventSetupImpl const&amp, edm::ActivityRegistry*, edm::ModuleCallingContext const*) () from /cvmfs/cms-ib.cern.ch/nweek-02618/slc7_amd64_gcc820/cms/cmssw/CMSSW_11_1_X_2020-03-01-2300/lib/slc7_amd64_gcc820/libFWCoreFramework.so

@makortel
Copy link
Contributor

makortel commented Mar 4, 2020

assign dqm

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 4, 2020

New categories assigned: dqm

@jfernan2,@andrius-k,@schneiml,@fioriNTU,@kmaeshima you have been requested to review this Pull request/Issue and eventually sign? Thanks

@Dr15Jones
Copy link
Contributor Author

The problem is here

https://github.com/cms-sw/cmssw/blob/master/DQM/L1TMonitor/src/L1TStage2EMTF.cc#L829-L830

where the underlying TH2Fs are being accessed without any locking.

@Dr15Jones
Copy link
Contributor Author

assign dqm, l1

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 4, 2020

New categories assigned: l1

@benkrikler,@rekovic you have been requested to review this Pull request/Issue and eventually sign? Thanks

@Dr15Jones
Copy link
Contributor Author

Doing

 git grep getTH | awk -F ':' '{print $1}' | sort -u | wc

gives 266 files that should be checked.

schneiml added a commit to schneiml/cmssw that referenced this issue Mar 4, 2020
@schneiml
Copy link
Contributor

schneiml commented Mar 4, 2020

I opened a PR to change this specific module back to edm::one, that should prevent the crash, but that code should definitely be reviewed.

Regarding all the other getTH1 calls: The majority are in harvesting, where getTH1 is discouraged, but safe and also unavoidable in many cases (as of today).

The challenge is to systematically find all getTH1 calls that come out of DQM*EDAnalyzers (even from DQMOneEDAnalyzer it is not in general safe to call getTH1, since other modules might access the same object -- this is rare but possible now). The relevant awk for that is beyond my level of ad-hoc code analysis, I did some experiments with the DXR code search database but even there it takes a rather big join...

@Dr15Jones
Copy link
Contributor Author

After filtering out all known 'safe' ED module types (which include inheriting from DQMEDHarvester, I still find 222 classes which are calling getTH

See

DQMOffline/PFTau/interface/CandidateBenchmark.h
DQMOffline/PFTau/interface/METBenchmark.h
DQMOffline/PFTau/interface/MatchCandidateBenchmark.h
DQMOffline/PFTau/interface/MatchMETBenchmark.h
DQMOffline/PFTau/interface/PFCandidateBenchmark.h
DQMOffline/PFTau/interface/PFCandidateManager.h
DQMOffline/PFTau/interface/PFCandidateMonitor.h
DQMOffline/PFTau/interface/PFJetMonitor.h
DQMOffline/PFTau/interface/PFMETMonitor.h
DQMOffline/PFTau/plugins/BenchmarkAnalyzer.h
DQMOffline/PFTau/plugins/CandidateBenchmarkAnalyzer.h
DQMOffline/PFTau/plugins/METBenchmarkAnalyzer.h
DQMOffline/PFTau/plugins/MatchMETBenchmarkAnalyzer.h
DQMOffline/PFTau/plugins/PFCandidateBenchmarkAnalyzer.h
DQMOffline/PFTau/plugins/PFCandidateManagerAnalyzer.h
RecoParticleFlow/Benchmark/interface/BenchmarkTree.h
DQM/CSCMonitorModule/plugins/CSCMonitorObject.h
DQM/CTPPS/plugins/CTPPSPixelDQMSource.cc
DQM/HcalCommon/interface/Container1D.h
DQM/HcalCommon/interface/Container2D.h
DQM/HcalCommon/interface/ContainerProf1D.h
DQM/HcalCommon/interface/Container2D.h
DQM/HcalCommon/interface/ContainerProf2D.h
DQM/HcalCommon/interface/ContainerSingle1D.h
DQM/HcalCommon/interface/ContainerSingleProf1D.h
DQM/HcalCommon/interface/ContainerSingle2D.h
DQM/HcalCommon/interface/ContainerSingleProf2D.h
DQMServices/Components/plugins/DQMMessageLogger.h
DQMServices/Components/plugins/DQMStoreStats.h
DQMServices/Components/plugins/DQMStoreStats.h
DQMServices/Components/plugins/DQMStoreStats.h
DQMServices/Components/plugins/DQMStoreStats.h
DQM/DTMonitorClient/src/DTLocalTriggerEfficiencyTest.h
DQM/DTMonitorClient/src/DTLocalTriggerSynchTest.h
DQM/DTMonitorClient/src/DTTriggerEfficiencyTest.h
DQM/DataScouting/plugins/DiJetVarAnalyzer.h
DQM/HcalTasks/interface/DigiRunSummary.h
DQM/EcalPreshowerMonitorClient/interface/ESPedestalClient.h
DQM/CTPPS/plugins/ElasticPlotDQMSource.cc
DQMOffline/EGamma/interface/ElectronDqmAnalyzerBase.h
DQMOffline/EGamma/plugins/ElectronAnalyzer.h
DQMOffline/EGamma/plugins/ElectronGeneralAnalyzer.h
DQMOffline/EGamma/plugins/ElectronTagProbeAnalyzer.h
Validation/RecoEgamma/plugins/ElectronMcFakeValidator.h
Validation/RecoEgamma/plugins/ElectronMcMiniAODSignalValidator.h
Validation/RecoEgamma/plugins/ElectronMcSignalValidator.h
DQMOffline/EGamma/plugins/ElectronOfflineClient.h
Validation/RecoEgamma/plugins/ElectronMcFakePostValidator.h
Validation/RecoEgamma/plugins/ElectronMcMiniAODSignalPostValidator.h
Validation/RecoEgamma/plugins/ElectronMcSignalPostValidator.h
Validation/RecoEgamma/plugins/ElectronMcFakePostValidator.h
Validation/RecoEgamma/plugins/ElectronMcMiniAODSignalPostValidator.h
Validation/RecoEgamma/plugins/ElectronMcSignalPostValidator.h
DQMOffline/Trigger/plugins/GeneralHLTOffline.cc
DQMOffline/Trigger/interface/HLTInclusiveVBFSource.h
HLTriggerOffline/Btag/interface/HLTVertexPerformanceAnalyzer.h
DQM/HLXMonitor/interface/HLXMonitor.h
DQM/HcalTasks/interface/HcalOnlineHarvesting.h
DQM/SiPixelPhase1Common/interface/SiPixelPhase1Base.h
CalibTracker/SiStripChannelGain/interface/SiStripGainsPCLWorker.h
DQM/HLTEvF/plugins/DaqTestHistograms.cc
DQM/HLTEvF/plugins/LumiMonitor.cc
DQM/HLTEvF/plugins/PSMonitor.cc
DQM/HLTEvF/plugins/TriggerBxMonitor.cc
DQM/HLTEvF/plugins/TriggerBxVsOrbitMonitor.cc
DQM/HLTEvF/plugins/TriggerRatesMonitor.cc
DQM/SiStripCommissioningClients/interface/ApvTimingHistograms.h
DQM/SiStripCommissioningClients/interface/CalibrationHistograms.h
DQM/SiStripCommissioningClients/interface/DaqScopeModeHistograms.h
DQM/SiStripCommissioningClients/interface/FastFedCablingHistograms.h
DQM/SiStripCommissioningClients/interface/FedCablingHistograms.h
DQM/SiStripCommissioningClients/interface/FedTimingHistograms.h
DQM/SiStripCommissioningClients/interface/NoiseHistograms.h
DQM/SiStripCommissioningClients/interface/OptoScanHistograms.h
DQM/SiStripCommissioningClients/interface/PedestalsHistograms.h
DQM/SiStripCommissioningClients/interface/PedsFullNoiseHistograms.h
DQM/SiStripCommissioningClients/interface/PedsOnlyHistograms.h
DQM/SiStripCommissioningClients/interface/SamplingHistograms.h
DQM/SiStripCommissioningClients/interface/VpspScanHistograms.h
DQM/SiStripCommissioningDbClients/interface/ApvTimingHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/CalibrationHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/CommissioningHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/DaqScopeModeHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/FastFedCablingHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/FineDelayHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/LatencyHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/NoiseHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/OptoScanHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/PedestalsHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/PedsFullNoiseHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/PedsOnlyHistosUsingDb.h
DQM/SiStripCommissioningDbClients/interface/VpspScanHistosUsingDb.h
DQM/SiStripMonitorHardware/interface/CMHistograms.hh
DQM/SiStripMonitorHardware/interface/FEDHistograms.hh
DQM/SiStripMonitorHardware/interface/SPYHistograms.h
DQMOffline/Muon/interface/DiMuonHistograms.h
DQMOffline/RecoB/interface/TrackIPHistograms.h
DQMServices/Demo/plugins/DemoGlobalDQMEDAnalyzer.cc
DQMServices/Demo/test/TestDQMEDAnalyzer.cc
FWCore/Skeletons/scripts/mkTemplates/DQMEDAnalyzer/DQMEDAnalyzer.cc
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
MuonAnalysis/MomentumScaleCalibration/plugins/Histograms.h
Validation/DTRecHits/plugins/DTRecHitQuality.h
Validation/DTRecHits/plugins/DTSegment2DQuality.h
Validation/DTRecHits/plugins/DTSegment2DSLPhiQuality.h
Validation/DTRecHits/plugins/DTSegment4DQuality.h
Validation/HGCalValidation/interface/HGCalValidator.h
Validation/HGCalValidation/plugins/CaloParticleValidation.cc
Validation/HGCalValidation/plugins/TICLPFValidation.cc
Validation/RecoTrack/interface/MultiTrackValidator.h
DQMOffline/JetMET/interface/JetAnalyzer_HeavyIons.h
DQMOffline/JetMET/interface/JetAnalyzer_HeavyIons_matching.h
DQMOffline/Trigger/interface/JetMETHLTOfflineSource.h
DQM/L1TMonitor/interface/L1TCSCTF.h
DQM/L1TMonitor/interface/L1TDTTF.h
DQM/L1TMonitor/interface/L1TMP7ZeroSupp.h
DQMOffline/L1Trigger/interface/L1TStage2CaloLayer2Offline.h
DQM/L1TMonitor/interface/L1TStage2EMTF.h
DQM/L1TMonitor/interface/L1TStage2MuonComp.h
DQM/L1TMonitor/interface/L1TStage2RegionalMuonCandComp.h
DQM/L1TMonitor/interface/L1TStage2uGTCaloLayer2Comp.h
DQMOffline/L1Trigger/interface/L1TTauOffline.h
Validation/EventGenerator/plugins/MBUEandQCDValidation.h
Validation/MuonME0Validation/interface/ME0DigisValidation.h
DQM/EcalCommon/interface/MESetChannel.h
DQM/EcalCommon/interface/MESetDet0D.h
DQM/EcalCommon/interface/MESetDet1D.h
DQM/EcalCommon/interface/MESetDet2D.h
DQM/EcalCommon/interface/MESetEcal.h
DQM/EcalCommon/interface/MESetMulti.h
DQM/EcalCommon/interface/MESetNonObject.h
DQM/EcalCommon/interface/MESetProjection.h
DQM/EcalCommon/interface/MESetTrend.h
DQM/EcalCommon/interface/MESetDet1D.h
DQM/EcalCommon/interface/MESetDet2D.h
DQM/EcalCommon/interface/MESetDet0D.h
DQM/EcalCommon/interface/MESetDet1D.h
DQM/EcalCommon/interface/MESetDet2D.h
DQM/EcalCommon/interface/MESetEcal.h
DQM/EcalCommon/interface/MESetProjection.h
DQM/EcalCommon/interface/MESetTrend.h
DQM/EcalCommon/interface/MESetNonObject.h
DQM/EcalCommon/interface/MESetProjection.h
DQM/EcalCommon/interface/MESetTrend.h
DQMOffline/Muon/interface/MuonIsolationDQM.h
DQMOffline/Muon/interface/MuonPFAnalyzer.h
Validation/RecoMuon/plugins/MuonTrackValidator.h
DQM/TrackingMonitor/src/PackedCandidateTrackValidator.cc
DQM/SiStripCommissioningSources/interface/PedsFullNoiseTask.h
DQM/PhysicsObjectsMonitoring/interface/PhysicsObjectsMonitor.h
Validation/RecoVertex/interface/PrimaryVertexAnalyzer4PUSlimmed.h
DQM/TrackingMonitor/src/PrimaryVertexResolution.cc
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
RecoParticleFlow/PFClusterProducer/interface/PFRecHitQTests.h
DQM/Physics/src/QcdPhotonsDQM.h
DQM/Physics/src/QcdUeDQM.h
DQM/RPCMonitorClient/interface/RPCMonitorRaw.h
Validation/RPCRecHits/interface/RPCRecHitValid.h
DQM/DataScouting/interface/ScoutingAnalyzerBase.h
DQM/DataScouting/plugins/AlphaTVarAnalyzer.h
DQM/DataScouting/plugins/DiJetVarAnalyzer.h
DQM/DataScouting/plugins/RazorVarAnalyzer.h
DQM/DataScouting/plugins/ScoutingTestAnalyzer.h
DQM/SiPixelMonitorCluster/interface/SiPixelClusterSource.h
CalibTracker/SiPixelGainCalibration/plugins/SiPixelGainCalibrationAnalysis.h
CalibTracker/SiPixelIsAliveCalibration/src/SiPixelIsAliveCalibration.cc
CalibTracker/SiPixelSCurveCalibration/interface/SiPixelSCurveCalibrationAnalysis.h
DQM/SiStripMonitorSummary/interface/SiStripApvGainsDQM.h
DQM/SiStripMonitorSummary/interface/SiStripBackPlaneCorrectionDQM.h
DQM/SiStripMonitorSummary/interface/SiStripApvGainsDQM.h
DQM/SiStripMonitorSummary/interface/SiStripBackPlaneCorrectionDQM.h
DQM/SiStripMonitorSummary/interface/SiStripCablingDQM.h
DQM/SiStripMonitorSummary/interface/SiStripLorentzAngleDQM.h
DQM/SiStripMonitorSummary/interface/SiStripNoisesDQM.h
DQM/SiStripMonitorSummary/interface/SiStripPedestalsDQM.h
DQM/SiStripMonitorSummary/interface/SiStripQualityDQM.h
DQM/SiStripMonitorSummary/interface/SiStripThresholdDQM.h
DQM/SiStripMonitorSummary/interface/SiStripCablingDQM.h
CalibTracker/SiStripChannelGain/plugins/SiStripGainFromCalibTree.cc
CalibTracker/SiStripChannelGain/plugins/SiStripGainFromData.cc
DQM/SiStripMonitorSummary/interface/SiStripLorentzAngleDQM.h
DQM/SiStripMonitorCluster/interface/SiStripMonitorCluster.h
DQM/SiStripMonitorTrack/interface/SiStripMonitorTrack.h
DQM/SiStripMonitorSummary/interface/SiStripPedestalsDQM.h
DQM/SiStripMonitorSummary/interface/SiStripQualityDQM.h
DQMOffline/CalibTracker/plugins/SiStripQualityHotStripIdentifierRoot.h
DQMOffline/CalibTracker/src/StatisticsFilter.cc
DQMOffline/Alignment/interface/TkAlCaRecoMonitor.h
DQM/CTPPS/plugins/TotemRPDQMSource.cc
DQMOffline/Muon/interface/SegmentTrackAnalyzer.h
Validation/RecoMuon/src/MuonTrackAnalyzer.h
DQM/TrackingMonitor/interface/V0Monitor.h
DQMOffline/RecoB/plugins/PrimaryVertexMonitor.h
DQM/HcalTasks/interface/ZDCTask.h
Validation/HcalHits/interface/ZdcSimHitStudy.h

@makortel
Copy link
Contributor

makortel commented Mar 4, 2020

The challenge is to systematically find all getTH1 calls that come out of DQM*EDAnalyzers (even from DQMOneEDAnalyzer it is not in general safe to call getTH1, since other modules might access the same object -- this is rare but possible now). The relevant awk for that is beyond my level of ad-hoc code analysis, I did some experiments with the DXR code search database but even there it takes a rather big join...

I don't think there is a better way than to go through Chris' list by hand and classify the files/classes depending on the action taken (because likely something needs to be done anyway for most/all of them eventually).

@schneiml
Copy link
Contributor

schneiml commented Mar 4, 2020

@Dr15Jones Thanks, that list actually looks rather manageable. One more thing to exclude are legacy EDAnalyzers (I see some familiar names in the list), these also only run in harvesting and have proven to be notoriously hard to change at all (e.g. the Commissioning code).

Now the next hard part is what to do with code like the L1T module here, we'll need to pass that down to the subsystems in many cases since the usages require some domain knowledge to understand. We could just replace a complete replacement for getTH1 (e.g. callback based, like we have in booking now), but I'd prefer if we could get rid of all non-trivial operations on TH1 during filling time (to not block our path towards e.g. moving away from ROOT histograms).

@Dr15Jones
Copy link
Contributor Author

One more thing to exclude are legacy EDAnalyzers

those were excluded from the list I made.

@schneiml
Copy link
Contributor

@Dr15Jones ah ok. Though, e.g. everything in SiStripCommissioningClients is to my knowledge only used by legacy modules and will likely never be migrated (there is no need for it, the code only ever runs in private setups and does its task fine, I am told).

@davidlange6
Copy link
Contributor

davidlange6 commented Mar 12, 2020 via email

@schneiml
Copy link
Contributor

@davidlange6 From DQM side, they are. From EDM side, it might be required to change them to one modules with lumi transitions, but that is a trivial change.

And then, for that specific code the time horizon is probably LS3, where the respective hardware gets removed... but until then, it might still be required and there likely isn't any manpower for any nontrivial changes.

@Dr15Jones
Copy link
Contributor Author

@schneiml my simple filtering of the list removed code that was directly called by legacy EDAnalyzers but not code in helpers where the helper was then used by a module.

@jfernan2
Copy link
Contributor

Is this issue still alive?

@makortel
Copy link
Contributor

The problem mentioned in the description and #29096 (comment) appears to still be there

for (int hist_index = 0; hist_index < 5; ++hist_index) {
cscLCTTimingFrac[hist_index]->getTH2F()->Divide(cscLCTTiming[hist_index]->getTH2F(), cscTimingTot->getTH2F());
rpcHitTimingFrac[hist_index]->getTH2F()->Divide(rpcHitTiming[hist_index]->getTH2F(), rpcHitTimingTot->getTH2F());
}

@makortel
Copy link
Contributor

Although the segfault was circumvented in #29097 by making the module DQMOneEDAnalyzer.

@jfernan2
Copy link
Contributor

@jfernan2
Copy link
Contributor

ping @vukasinmilosevic

@vukasinmilosevic
Copy link
Contributor

vukasinmilosevic commented Dec 14, 2021

HI @jfernan2 we have someone taking a look into this. He is currently getting up to speed and will update us soon.

@cecilecaillol
Copy link
Contributor

@vukasinmilosevic Any news on this issue?

@vukasinmilosevic
Copy link
Contributor

Hi @cecilecaillol, I will check with the EMTF experts and get back at you asap.

@cecilecaillol
Copy link
Contributor

@vukasinmilosevic Did you get a chance to check with the EMTF experts?

@cecilecaillol
Copy link
Contributor

@vukasinmilosevic Do you have any update on this issue?

@vukasinmilosevic
Copy link
Contributor

Hi @cecilecaillol, apologies this fell down the list of tasks. I will follow up with relevant experts after the holiday period.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants