[MkFit] Per iteration / eta region track candidate scoring function #40490

osschar · 2023-01-11T23:56:12Z

PR description:

Allow track/candidate scoring function to be defined per iteration and eta region.
Configure through standard function catalog and JSON files.
Default is to pick the existing scoring function so there is no need to provide new JSON files until new scoring functions are actually implemented.
Definitions of function types used in standard function catalogs are moved from MkFitCore/interface/IterationConfig.h to FunctionTypes.h

The purpose of this PR is to allow further work on improvements of MKFit performance.

PR validation:

This is a technical change, there should be no changes in physics performance.

…d eta region. Configure through standard function catalog and JSON files.

cmsbuild · 2023-01-12T00:03:41Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40490/33667

This PR adds an extra 104KB to repository

cmsbuild · 2023-01-12T00:04:03Z

A new Pull Request was created by @osschar (Matevž Tadel) for master.

It involves the following packages:

RecoTracker/MkFitCMS (reconstruction)
RecoTracker/MkFitCore (reconstruction)

@cmsbuild, @mandrenguyen, @clacaputo can you please review it and eventually sign? Thanks.
@VourMa, @makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @missirol, @gpetruc, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

mmusich · 2023-01-12T08:09:23Z

type tracking

mmusich · 2023-01-12T08:09:29Z

please test

cmsbuild · 2023-01-12T09:22:32Z

-1

Failed Tests: HeaderConsistency RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e223d6/29933/summary.html
COMMIT: 57d737a
CMSSW: CMSSW_13_0_X_2023-01-11-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40490/29933/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

138.4

----- Begin Fatal Exception 12-Jan-2023 09:27:07 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 346512 lumi: 250 event: 243042266 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Prefetching for module DQMMessageLogger/'DQMMessageLogger'
   [3] Prefetching for module LogErrorHarvester/'logErrorHarvester'
   [4] Prefetching for module HcalNoiseInfoProducer/'hcalnoise'
   [5] Prefetching for module FastjetJetProducer/'ak4PFJets'
   [6] Prefetching for module PFLinker/'particleFlow'
   [7] Prefetching for module PFProducer/'particleFlowTmp'
   [8] Prefetching for module PFBlockProducer/'particleFlowBlock'
   [9] Prefetching for module PFElecTkProducer/'pfTrackElec'
   [10] Prefetching for module GsfTrackProducer/'electronGsfTracks'
   [11] Prefetching for module CkfTrackCandidateMaker/'electronCkfTrackCandidates'
   [12] Prefetching for module MeasurementTrackerEventProducer/'MeasurementTrackerEvent'
   [13] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [14] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [15] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [16] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [17] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [18] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_default_track_scorer_name' not found
----- End Fatal Exception -------------------------------------------------

138.5

----- Begin Fatal Exception 12-Jan-2023 09:27:20 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 346512 lumi: 250 event: 243042266 stream: 0
   [1] Running path 'dqmoffline_13_step'
   [2] Prefetching for module SMPDQM/'SMPDQM'
   [3] Prefetching for module MuonProducer/'muons'
   [4] Prefetching for module MuonIdProducer/'muons1stStep'
   [5] Prefetching for module DuplicateListMerger/'generalTracks'
   [6] Prefetching for module TrackProducer/'mergedDuplicateTracks'
   [7] Prefetching for module DuplicateTrackMerger/'duplicateTrackCandidates'
   [8] Prefetching for module TrackCollectionMerger/'preDuplicateMergingGeneralTracks'
   [9] Prefetching for module TrackCollectionMerger/'earlyGeneralTracks'
   [10] Prefetching for module TrackProducer/'initialStepTracks'
   [11] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidates'
   [12] Prefetching for module MkFitEventOfHitsProducer/'mkFitEventOfHits'
   [13] Prefetching for module MkFitSiPixelHitConverter/'mkFitSiPixelHits'
   [14] Prefetching for module SiPixelRecHitConverter/'siPixelRecHits'
   [15] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [16] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [17] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [18] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [19] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [20] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_default_track_scorer_name' not found
----- End Fatal Exception -------------------------------------------------

136.874

----- Begin Fatal Exception 12-Jan-2023 09:29:27 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 319450 lumi: 76 event: 105676750 stream: 0
   [1] Running path 'dqmoffline_10_step'
   [2] Prefetching for module SMPDQM/'SMPDQM'
   [3] Prefetching for module MuonProducer/'muons'
   [4] Prefetching for module MuonIdProducer/'muons1stStep'
   [5] Prefetching for module HBHEIsolatedNoiseReflagger/'hbhereco@cpu'
   [6] Prefetching for module TrackExtrapolator/'trackExtrapolator'
   [7] Prefetching for module DuplicateListMerger/'generalTracks'
   [8] Prefetching for module TrackProducer/'mergedDuplicateTracks'
   [9] Prefetching for module DuplicateTrackMerger/'duplicateTrackCandidates'
   [10] Prefetching for module TrackCollectionMerger/'preDuplicateMergingGeneralTracks'
   [11] Prefetching for module TrackCollectionMerger/'earlyGeneralTracks'
   [12] Prefetching for module TrackProducer/'initialStepTracks'
   [13] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidates'
   [14] Prefetching for module MkFitEventOfHitsProducer/'mkFitEventOfHits'
   [15] Prefetching for module MkFitSiPixelHitConverter/'mkFitSiPixelHits'
   [16] Prefetching for module SiPixelRecHitConverter/'siPixelRecHits'
   [17] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [18] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [19] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [20] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [21] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [22] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_default_track_scorer_name' not found
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

138.4138.4_PromptCollisions2021/step2_PromptCollisions2021.log
138.5138.5_ExpressCollisions2021/step2_ExpressCollisions2021.log
140.6140.6_RunHI2022/step2_RunHI2022.log

Expand to see more relval errors ...

AddOn Tests

hlt_mc_GRun

----- Begin Fatal Exception 12-Jan-2023 09:32:13 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 0
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module ReducedRecHitCollectionProducer/'reducedEcalRecHitsEB'
   [4] Prefetching for module InterestingDetIdCollectionProducer/'interestingEcalDetIdPFEB'
   [5] Prefetching for module PFECALSuperClusterProducer/'particleFlowSuperClusterECAL'
   [6] Prefetching for module RecoChargedRefCandidatePrimaryVertexSorter/'offlinePrimaryVertices'
   [7] Prefetching for module ChargedRefCandidateProducer/'trackRefsForJetsBeforeSorting'
   [8] Prefetching for module TrackWithVertexRefSelector/'trackWithVertexRefSelectorBeforeSorting'
   [9] Prefetching for module DuplicateListMerger/'generalTracks'
   [10] Prefetching for module TrackProducer/'mergedDuplicateTracks'
   [11] Prefetching for module DuplicateTrackMerger/'duplicateTrackCandidates'
   [12] Prefetching for module TrackCollectionMerger/'preDuplicateMergingGeneralTracks'
   [13] Prefetching for module TrackCollectionMerger/'earlyGeneralTracks'
   [14] Prefetching for module TrackProducer/'initialStepTracks'
   [15] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidates'
   [16] Prefetching for module MkFitEventOfHitsProducer/'mkFitEventOfHits'
   [17] Prefetching for module MkFitSiPixelHitConverter/'mkFitSiPixelHits'
   [18] Prefetching for module SiPixelRecHitConverter/'siPixelRecHits'
   [19] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [20] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [21] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [22] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [23] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [24] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_default_track_scorer_name' not found
----- End Fatal Exception -------------------------------------------------

hlt_mc_HIon

----- Begin Fatal Exception 12-Jan-2023 09:31:25 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 3
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module ReducedRecHitCollectionProducer/'reducedEcalRecHitsEB'
   [4] Prefetching for module InterestingDetIdCollectionProducer/'interestingEcalDetIdPFEB'
   [5] Prefetching for module PFECALSuperClusterProducer/'particleFlowSuperClusterECAL'
   [6] Prefetching for module RecoChargedRefCandidatePrimaryVertexSorter/'offlinePrimaryVertices'
   [7] Prefetching for module ChargedRefCandidateProducer/'trackRefsForJetsBeforeSorting'
   [8] Prefetching for module TrackWithVertexRefSelector/'trackWithVertexRefSelectorBeforeSorting'
   [9] Prefetching for module DuplicateListMerger/'generalTracks'
   [10] Prefetching for module TrackProducer/'mergedDuplicateTracks'
   [11] Prefetching for module DuplicateTrackMerger/'duplicateTrackCandidates'
   [12] Prefetching for module TrackCollectionMerger/'preDuplicateMergingGeneralTracks'
   [13] Prefetching for module TrackCollectionMerger/'earlyGeneralTracks'
   [14] Prefetching for module TrackProducer/'initialStepTracks'
   [15] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidates'
   [16] Prefetching for module MkFitEventOfHitsProducer/'mkFitEventOfHits'
   [17] Prefetching for module MkFitSiPixelHitConverter/'mkFitSiPixelHits'
   [18] Prefetching for module SiPixelRecHitConverter/'siPixelRecHits'
   [19] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [20] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [21] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [22] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [23] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [24] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_default_track_scorer_name' not found
----- End Fatal Exception -------------------------------------------------

hlt_mc_PIon

----- Begin Fatal Exception 12-Jan-2023 09:30:25 CET-----------------------
An exception of category 'StdException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 4 stream: 1
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module ReducedRecHitCollectionProducer/'reducedEcalRecHitsEB'
   [4] Prefetching for module InterestingDetIdCollectionProducer/'interestingEcalDetIdPFEB'
   [5] Prefetching for module PFECALSuperClusterProducer/'particleFlowSuperClusterECAL'
   [6] Prefetching for module RecoChargedRefCandidatePrimaryVertexSorter/'offlinePrimaryVertices'
   [7] Prefetching for module ChargedRefCandidateProducer/'trackRefsForJetsBeforeSorting'
   [8] Prefetching for module TrackWithVertexRefSelector/'trackWithVertexRefSelectorBeforeSorting'
   [9] Prefetching for module DuplicateListMerger/'generalTracks'
   [10] Prefetching for module TrackProducer/'mergedDuplicateTracks'
   [11] Prefetching for module DuplicateTrackMerger/'duplicateTrackCandidates'
   [12] Prefetching for module TrackCollectionMerger/'preDuplicateMergingGeneralTracks'
   [13] Prefetching for module TrackCollectionMerger/'earlyGeneralTracks'
   [14] Prefetching for module TrackProducer/'initialStepTracks'
   [15] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidates'
   [16] Prefetching for module MkFitEventOfHitsProducer/'mkFitEventOfHits'
   [17] Prefetching for module MkFitSiPixelHitConverter/'mkFitSiPixelHits'
   [18] Prefetching for module SiPixelRecHitConverter/'siPixelRecHits'
   [19] Prefetching for module JetCoreClusterSplitter/'siPixelClusters'
   [20] Prefetching for module PrimaryVertexProducer/'firstStepPrimaryVerticesPreSplitting'
   [21] Prefetching for module TrackProducer/'initialStepTracksPreSplitting'
   [22] Prefetching for module MkFitOutputConverter/'initialStepTrackCandidatesPreSplitting'
   [23] Prefetching for module MkFitProducer/'initialStepTrackCandidatesMkFitPreSplitting'
   [24] Calling method for EventSetup module MkFitIterationConfigESProducer/'initialStepTrackCandidatesMkFitConfigPreSplitting'
Exception Message:
A std::exception was thrown.
[json.exception.out_of_range.403] key 'm_default_track_scorer_name' not found
----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

osschar · 2023-01-12T17:46:56Z

Sigh, I wrongly assumed JSON parser wouldn't mind the missing members ... that they would just keep the default values.
I will prepare a parallel PR for the JSON files in the cms-data/RecoTracker-MkFit.

I apologize for the omission and noise.

…aming.

osschar · 2023-01-12T21:26:10Z

I have added the JSON changes: cms-data/RecoTracker-MkFit#11

slava77 · 2023-01-12T21:30:16Z

test parameters:

pull_request = Add [default_]track_scorer_name entries, goes with CMSSW #40490. cms-data/RecoTracker-MkFit#11

cmsbuild · 2023-01-12T21:31:58Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40490/33687

This PR adds an extra 20KB to repository

osschar · 2023-01-16T01:17:07Z

Thanks @smuzaffar ... I added the include.

cmsbuild · 2023-01-16T01:23:56Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40490/33735

This PR adds an extra 12KB to repository

cmsbuild · 2023-01-16T01:24:20Z

Pull request #40490 was updated. @cmsbuild, @mandrenguyen, @clacaputo can you please check and sign again.

slava77 · 2023-01-20T16:18:37Z

@cmsbuild please test

the differences are now understood and no further changes are planned to this PR, unless some are requested in the review

mmasciov · 2023-01-20T20:50:11Z

the differences are now understood and no further changes are planned to this PR, unless some are requested in the review

For completeness:

There are slight differences in physics performance visible in the MTV results for individual mkFit iterations introduced with this PR; e.g., in TTbar events with PU: http://uaf-10.t2.ucsd.edu/~mmasciov/MIC/PR40490/MTV_TTbarPU_mkFit_PR40490_EL8/
After further investigations, it became apparent that these slight differences are related to the presence of the fast-math flag in the MkFitCore buildfile, and to the fact that previously inline functions are now declared as regular functions. In fact, if one removes the fast-math flag from the MkFitCore buildfile, the corresponding MTV results show zero differences: http://uaf-10.t2.ucsd.edu/~mmasciov/MIC/PR40490/MTV_TTbarPU_mkFit_PR40490_EL8_nofastmathMkFitCoreOnly/

As the above-mentioned slight differences are related to compilation and not to the actual changes introduced by this PR, we would consider the PR itself validated in terms of performance.

@osschar @slava77 @mmusich

cmsbuild · 2023-01-20T21:39:03Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e223d6/30100/summary.html
COMMIT: d4121ef
CMSSW: CMSSW_13_0_X_2023-01-20-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40490/30100/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e223d6/30100/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e223d6/30100/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 1805 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3555479
DQMHistoTests: Total failures: 9407
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3546050
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

mandrenguyen · 2023-01-22T08:55:29Z

+reconstruction
Small changes to comparisons are due to change to compiler setting, as described in this comment

cmsbuild · 2023-01-22T08:55:55Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-01-22T11:11:38Z

+1

Differences in tha last test outputs are due to fastmath usage, see [MkFit] Per iteration / eta region track candidate scoring function #40490 (comment), and to the additional PRs built together, in particular Correct GEM edge calculation for TrackerMuons #40554

Allow track/candidate scoring function to be defined per iteration an…

57d737a

…d eta region. Configure through standard function catalog and JSON files.

cmsbuild added this to the CMSSW_13_0_X milestone Jan 11, 2023

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jan 11, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 12, 2023

cmsbuild added tests-started tracking and removed tests-pending labels Jan 12, 2023

cmsbuild added tests-rejected and removed tests-started labels Jan 12, 2023

Missing assignment for standalone config setup, consistent argument n…

7155ed1

…aming.

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Jan 12, 2023

osschar mentioned this pull request Jan 12, 2023

Add [default_]track_scorer_name entries, goes with CMSSW #40490. cms-data/RecoTracker-MkFit#11

Merged

cmsbuild added the requires-external label Jan 12, 2023

cmsbuild added tests-rejected and removed tests-started labels Jan 13, 2023

Include FunctionTypes.h to make header self consistent.

d4121ef

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Jan 16, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 16, 2023

cmsbuild added tests-started and removed tests-pending labels Jan 20, 2023

cmsbuild added tests-approved and removed tests-started labels Jan 20, 2023

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Jan 22, 2023

cmsbuild added orp-approved and removed orp-pending labels Jan 22, 2023

cmsbuild merged commit 1d62219 into cms-sw:master Jan 22, 2023

cmsbuild mentioned this pull request Jan 23, 2023

Implement a simple vector add test with alpaka #40580

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MkFit] Per iteration / eta region track candidate scoring function #40490

[MkFit] Per iteration / eta region track candidate scoring function #40490

osschar commented Jan 11, 2023

cmsbuild commented Jan 12, 2023

cmsbuild commented Jan 12, 2023

mmusich commented Jan 12, 2023

mmusich commented Jan 12, 2023

cmsbuild commented Jan 12, 2023

osschar commented Jan 12, 2023

osschar commented Jan 12, 2023

slava77 commented Jan 12, 2023

cmsbuild commented Jan 12, 2023

osschar commented Jan 16, 2023

cmsbuild commented Jan 16, 2023

cmsbuild commented Jan 16, 2023

slava77 commented Jan 20, 2023

mmasciov commented Jan 20, 2023 •

edited

Loading

cmsbuild commented Jan 20, 2023

mandrenguyen commented Jan 22, 2023

cmsbuild commented Jan 22, 2023

perrotta commented Jan 22, 2023

[MkFit] Per iteration / eta region track candidate scoring function #40490

[MkFit] Per iteration / eta region track candidate scoring function #40490

Conversation

osschar commented Jan 11, 2023

PR description:

PR validation:

cmsbuild commented Jan 12, 2023

cmsbuild commented Jan 12, 2023

mmusich commented Jan 12, 2023

mmusich commented Jan 12, 2023

cmsbuild commented Jan 12, 2023

RelVals

RelVals-INPUT

AddOn Tests

osschar commented Jan 12, 2023

osschar commented Jan 12, 2023

slava77 commented Jan 12, 2023

cmsbuild commented Jan 12, 2023

osschar commented Jan 16, 2023

cmsbuild commented Jan 16, 2023

cmsbuild commented Jan 16, 2023

slava77 commented Jan 20, 2023

mmasciov commented Jan 20, 2023 • edited Loading

cmsbuild commented Jan 20, 2023

Comparison Summary

mandrenguyen commented Jan 22, 2023

cmsbuild commented Jan 22, 2023

perrotta commented Jan 22, 2023

mmasciov commented Jan 20, 2023 •

edited

Loading