Missed bounds check optimization

[davidbolvansky]

Bugzilla Link	47172
Version	trunk
OS	Linux
CC	@efriedma-quic,@fhahn,@jrmuizel,@nikic

Extended Description

From rust-lang github (rust-lang/rust#75525):

#include <stddef.h>
#include <stdint.h>


uint8_t f1(size_t idx) {
    if (idx < 8) {
        // it's ok that this has a bounds check
        return (idx - 1) < 10;
    } else {
        return 0;
    }
}

Clang:

f1:                                     # @​f1
        cmp     rdi, 8
        setb    cl
        add     rdi, -1
        cmp     rdi, 10
        setb    al
        and     al, cl
        ret

GCC:

f1:
        sub     rdi, 1
        cmp     rdi, 6
        setbe   al
        ret

Modified test case:

uint8_t f2(size_t idx) {
    if (idx <= 9) {
       return (idx - 1) > 10;
    } else {
        return 0;
    }
}

Clang:

f2:                                     # @​f2
        cmp     rdi, 10
        setb    cl
        add     rdi, -1
        cmp     rdi, 10
        seta    al
        and     al, cl
        ret

GCC:

f2:
        test    rdi, rdi
        sete    al
        ret

https://godbolt.org/z/vahzbo

[fhahn]

Variants of this should be able to be handled using the approach suggested here https://reviews.llvm.org/D84547

Although in the C source, idx - 1 can wrap. I'm not entirely sure about the rust case, but I assume there might be some wrapping flags?

[efriedma-quic]

gcc is optimizing assuming the subtraction can wrap.

rustc generally doesn't generate nsw/nuw flags in the frontend; integer overflow is defined to either wrap or trap.

[davidbolvansky]

One more test case for CE pass:

uint8_t f3(size_t idx) {
    if (idx <= 9) {
       return (idx + 1) > 16;
    } else {
        return 0;
    }
}

Clang:

f3:                                    
        cmp     rdi, 10
        setb    cl
        add     rdi, 1
        cmp     rdi, 16
        seta    al
        and     al, cl
        ret

GCC:

f3:
        xor     eax, eax
        ret

[fhahn]

Looks like Clang trunk optimizes this now as expected: https://godbolt.org/z/a55674xhe