[clang-tidy] LLVM 16 will promote suppressed warnings to errors when C++20 is enabled

[tiagomacarios]

Repro:

$ cat a.cpp
#define A 1
#define A

void _()
{
    int i = 42;
    int ii, iii;
}

$ clang-tidy-16 a.cpp --checks=-*,readability-isolate-declaration --warnings-as-errors=* -- -std=c++17  -Wno-macro-redefined
1 warning generated.
D:\repro\a.cpp:7:5: error: multiple declarations in a single statement reduces readability [readability-isolate-declaration,-warnings-as-errors]
    int ii, iii;
    ^~~~~~~~~~~~
1 warning treated as error

$ clang-tidy-16 a.cpp --checks=-*,readability-isolate-declaration --warnings-as-errors=* -- -std=c++20 -Wno-macro-redefined
2 warnings generated.
D:\repro\a.cpp:2:9: error: 'A' macro redefined [clang-diagnostic-macro-redefined,-warnings-as-errors]
#define A
        ^
D:\repro\a.cpp:1:9: note: previous definition is here
#define A 1
        ^
D:\repro\a.cpp:7:5: error: multiple declarations in a single statement reduces readability [readability-isolate-declaration,-warnings-as-errors]
    int ii, iii;
    ^~~~~~~~~~~~
2 warnings treated as errors

$ clang-tidy-15 a.cpp --checks=-*,readability-isolate-declaration --warnings-as-errors=* -- -std=c++20 -Wno-macro-redefined
2 warnings generated.
D:\repro\a.cpp:7:5: error: multiple declarations in a single statement reduces readability [readability-isolate-declaration,-warnings-as-errors]
    int ii, iii;
    ^~~~~~~~~~~~
Suppressed 1 warnings (1 with check filters).
1 warning treated as error

$ clang-tidy-15 a.cpp --checks=-*,readability-isolate-declaration --warnings-as-errors=* -- -std=c++17 -Wno-macro-redefined
1 warning generated.
D:\repro\a.cpp:7:5: error: multiple declarations in a single statement reduces readability [readability-isolate-declaration,-warnings-as-errors]
    int ii, iii;
    ^~~~~~~~~~~~
1 warning treated as error

I see that there are other similar issues, but none of them mentions C++20:
#61520, #59588, and #59589

Will the fix proposed in https://reviews.llvm.org/D146520 also work here? Or is this something else?

[llvmbot]

@llvm/issue-subscribers-c-20

[tiagomacarios]

@EugeneZelenko can you tag this clang-tidy?

[llvmbot]

@llvm/issue-subscribers-clang-tidy

[carlosgalvezp]

Handling of warnings as errors is working exactly as expected - all warnings are enabled as errors since you use -warning-as-errors='*'.

The real problem is that on C++20, -Wno-macro-redefined does not seem to disable the warning in clang-tidy (it does in clang).

[PiotrZSL]

Issue with C++20 most probably is related to https://discourse.llvm.org/t/rfc-expand-modular-headers-ppcallbacks-problem-in-c-20/71628

[PiotrZSL]

Looks like 9644368 is a partial fix, it fixes handling of --warnings-as-errors. Other workaround is here: https://reviews.llvm.org/D156024