Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RECONSTRUCTION] [CLANG] Loop not vectorized warnings #41624

Closed
aandvalenzuela opened this issue May 11, 2023 · 12 comments · Fixed by #41730
Closed

[RECONSTRUCTION] [CLANG] Loop not vectorized warnings #41624

aandvalenzuela opened this issue May 11, 2023 · 12 comments · Fixed by #41730

Comments

@aandvalenzuela
Copy link
Contributor

aandvalenzuela commented May 11, 2023

Hello,

I am opening a separate issue for what we started discussing in #41616.

CLANG IBs started showing some warnings reporting loop not vectorized as of the update to llvm16.
In concrete, for the reconstruction module RecoTracker/MkFitCore:

>> Compiling  /pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/tmp/BUILDROOT/189e8911dd83c5e66aafe1fe240371ea/opt/cmssw/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_CLANG_X_2023-05-09-2300/src/RecoTracker/MkFitCore/src/radix_sort.cc
/pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/external/llvm/16.0.3-5d1904f3c7b5c53b7522ba0858baec44/bin/clang++ -c -DGNU_GCC -D_GNU_SOURCE -DTBB_USE_GLIBCXX_VERSION=110201 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DCMSSW_GIT_HASH='CMSSW_13_2_CLANG_X_2023-05-09-2300' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_13_2_CLANG_X_2023-05-09-2300' -I/pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/tmp/BUILDROOT/189e8911dd83c5e66aafe1fe240371ea/opt/cmssw/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_CLANG_X_2023-05-09-2300/src -I/pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/external/pcre/8.43-5dcc901acc02f624b22dd9840b2357e8/include -I/pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/external/bz2lib/1.0.6-2c1f18484cb66c30aba7929f2be5e7d4/include -isystem/pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/lcg/root/6.26.11-9720c7273eac1c892ddadde973cf1965/include -isystem/pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/external/tbb/v2021.8.0-4e779f195a25a0aba119b27519937ba0/include -I/pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/external/xz/5.2.5-83d0a00b575efd1701e07bedf7977343/include -I/pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/external/zlib/1.2.11-3dfb2715f3608466b74431b80eb9d788/include -I/pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/external/json/3.10.2-a6d86565b09ec3d0e02bf7b52c31bbfc/include -O2 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++17 -ftree-vectorize -Werror=array-bounds -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -fuse-ld=bfd -msse3 -felide-constructors -fmessage-length=0 -Wall -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-deprecated-copy -Wno-unused-parameter -Wunused -Wparentheses -Wno-deprecated -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-c99-extensions -Wno-c++11-narrowing -D__STRICT_ANSI__ -Wno-unused-private-field -Wno-unknown-pragmas -Wno-unused-command-line-argument -Wno-unknown-warning-option -ftemplate-depth=512 -Wno-error=potentially-evaluated-expression -Wno-tautological-type-limit-compare -fsized-deallocation --gcc-toolchain=/pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/el8_amd64_gcc11/external/gcc/11.2.1-f9b9dfdd886f71cd63f5538223d8f161 -Ofast -fno-reciprocal-math -mrecip=none -DBOOST_DISABLE_ASSERTS -fopenmp-simd  -fPIC    -MMD -MF tmp/el8_amd64_gcc11/src/RecoTracker/MkFitCore/src/RecoTrackerMkFitCore/radix_sort.cc.d /pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/tmp/BUILDROOT/189e8911dd83c5e66aafe1fe240371ea/opt/cmssw/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_CLANG_X_2023-05-09-2300/src/RecoTracker/MkFitCore/src/radix_sort.cc -o tmp/el8_amd64_gcc11/src/RecoTracker/MkFitCore/src/RecoTrackerMkFitCore/radix_sort.cc.o
  /pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/tmp/BUILDROOT/189e8911dd83c5e66aafe1fe240371ea/opt/cmssw/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_CLANG_X_2023-05-09-2300/src/RecoTracker/MkFitCore/src/PropagationMPlex.cc:290:8: warning: loop not vectorized: the optimizer was unable to perform the requested transformation; the transformation might be disabled or specified as part of an unsupported transformation ordering [-Wpass-failed=transform-warning]
   void helixAtRFromIterativeCCSFullJac(const MPlexLV& inPar,
       ^
  /pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/tmp/BUILDROOT/189e8911dd83c5e66aafe1fe240371ea/opt/cmssw/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_CLANG_X_2023-05-09-2300/src/RecoTracker/MkFitCore/src/MkFinder.cc:248:18: warning: loop not vectorized: the optimizer was unable to perform the requested transformation; the transformation might be disabled or specified as part of an unsupported transformation ordering [-Wpass-failed=transform-warning]
   void MkFinder::selectHitIndices(const LayerOfHits &layer_of_hits, const int N_proc) {
                 ^
  /pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/tmp/BUILDROOT/189e8911dd83c5e66aafe1fe240371ea/opt/cmssw/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_CLANG_X_2023-05-09-2300/src/RecoTracker/MkFitCore/src/MkFinder.cc:248:18: warning: loop not vectorized: the optimizer was unable to perform the requested transformation; the transformation might be disabled or specified as part of an unsupported transformation ordering [-Wpass-failed=transform-warning]
   /pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/tmp/BUILDROOT/189e8911dd83c5e66aafe1fe240371ea/opt/cmssw/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_CLANG_X_2023-05-09-2300/src/RecoTracker/MkFitCore/src/MkFinder.cc:693:18: warning: loop not vectorized: the optimizer was unable to perform the requested transformation; the transformation might be disabled or specified as part of an unsupported transformation ordering [-Wpass-failed=transform-warning]
   void MkFinder::addBestHit(const LayerOfHits &layer_of_hits, const int N_proc, const FindingFoos &fnd_foos) {
                 ^
  /pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/tmp/BUILDROOT/189e8911dd83c5e66aafe1fe240371ea/opt/cmssw/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_CLANG_X_2023-05-09-2300/src/RecoTracker/MkFitCore/src/MkFinder.cc:905:18: warning: loop not vectorized: the optimizer was unable to perform the requested transformation; the transformation might be disabled or specified as part of an unsupported transformation ordering [-Wpass-failed=transform-warning]
   void MkFinder::findCandidates(const LayerOfHits &layer_of_hits,
                 ^
  /pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/tmp/BUILDROOT/189e8911dd83c5e66aafe1fe240371ea/opt/cmssw/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_CLANG_X_2023-05-09-2300/src/RecoTracker/MkFitCore/src/MkFinder.cc:1151:18: warning: loop not vectorized: the optimizer was unable to perform the requested transformation; the transformation might be disabled or specified as part of an unsupported transformation ordering [-Wpass-failed=transform-warning]
   void MkFinder::findCandidatesCloneEngine(const LayerOfHits &layer_of_hits,
                 ^
  /pool/condor/dir_43945/jenkins/workspace/build-any-ib/w/tmp/BUILDROOT/189e8911dd83c5e66aafe1fe240371ea/opt/cmssw/el8_amd64_gcc11/cms/cmssw/CMSSW_13_2_CLANG_X_2023-05-09-2300/src/RecoTracker/MkFitCore/src/MkFinder.cc:1151:18: warning: loop not vectorized: the optimizer was unable to perform the requested transformation; the transformation might be disabled or specified as part of an unsupported transformation ordering [-Wpass-failed=transform-warning]
 1 warning generated.
6 warnings generated.

I have seen in older PRs that sometimes this has been fixed by using a pragma statement, others inlining the function (which I believe it won't work here, but correct me if I am wrong) and in other cases the original authors took care of rearranging the code. Any hints on what can we do in those cases?

Many thanks! :)
Andrea.

FYI, @smuzaffar

@aandvalenzuela
Copy link
Contributor Author

assign reconstruction

@aandvalenzuela
Copy link
Contributor Author

FYI, @slava77

@cmsbuild
Copy link
Contributor

New categories assigned: reconstruction

@mandrenguyen,@clacaputo you have been requested to review this Pull request/Issue and eventually sign? Thanks

@cmsbuild
Copy link
Contributor

A new Issue was created by @aandvalenzuela Andrea Valenzuela.

@Dr15Jones, @perrotta, @dpiparo, @rappoccio, @makortel, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

@aandvalenzuela
Copy link
Contributor Author

This issue is rather urgent to be able to integrate llvm 16 in cmssw master IBs.

@slava77
Copy link
Contributor

slava77 commented May 11, 2023

This issue is rather urgent to be able to integrate llvm 16 in cmssw master IBs.

do we really require zero warnings to be able to use a compiler?
What if the warnings are advisory as apparently in this case?

@srlantz reminded me that different loops had different success in vectorization using different compilers.
mkfit was initially developed with icc and it performed (still does?) much better than gcc in ability to vectorize.
Also, at least in some cases pragmas to vectorize were still added with outlook to the future: some improved compiler variant could vectorize.
So, these are "best effort" pragmas, rather than "must do" kind.

Is there a pragma or compiler macro that would instruct llvm here to not issue a warning for the "best effort" cases like here?

@slava77
Copy link
Contributor

slava77 commented May 11, 2023

type tracking

@smuzaffar
Copy link
Contributor

smuzaffar commented May 11, 2023

do we really require zero warnings to be able to use a compiler?

yes @slava77 , currently bot rejects the PR (and do not run tests) if it finds warnings in the file changed by the PR. So we need to clean then warnings otherwise any future change in these files will be rejected.

What if the warnings are advisory as apparently in this case?

If itis really needed, we can teach bot to ignore some advisory warnings

@slava77
Copy link
Contributor

slava77 commented May 11, 2023

I searched on the web and found https://useyourloaf.com/blog/disabling-clang-compiler-warnings/

#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wunused-variable"

int a;

#pragma clang diagnostic pop

I guess we can add it in mkFit.

@smuzaffar
how do I compile with llvm to test? Should I just use the CMSSW_13_2_CLANG_X IB and scram b there?

@dan131riley
Copy link

@slava77 yes you can just use a recent CLANG IB, I'm looking to see if there are any better options to get those loops to vectorize. So far unrolling the loops doesn't help.

@dan131riley
Copy link

#pragma clang diagnostic ignored "-Wunused-variable"

It appears that if you have a simd pragma it unconditionally turns on the warning that vectorization failed and it can't be turned off--or at least I couldn't find a flag to turn it off. From the message you might think ignoring "-Wpass-failed=transform-warning" or "-Wpass-failed" might work, but neither suppressed the warning message. So I guess it'll have to be conditioned on whether "clang" is defined.

@dan131riley
Copy link

Fix submitted in PR #41730

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants