'Not supported' exception raised when running AD program.

[Enigmatisms]

Hi, I've been trying to make my renderer differentiable and it winds up having this exception:

[I 10/09/23 16:43:02.334 69589] [auto_diff.cpp:visit@869] 
"unary_op_type_name(stmt->op_type)": "cast_bits"
[E 10/09/23 16:43:02.334 69589] [auto_diff.cpp:visit@870] Not supported.

I am using Taichi 1.6.0 on ubuntu 20.04 (with CUDA backend). I am just not sure what could trigger this problem since in my code:

with ti.ad.Tape(loss=loss):
    rdr.render(eye_start, eye_end, lit_start, lit_end, max_bounce, max_depth)
    loss_backward(rdr, gt, loss, 1. / (rdr.w * rdr.h))

would fail but once the rdr.render is moved outside of Tape everything works (but without valid gradient update). The code I am using is this: Enigmatisms/AdaPT which is too complex to have a minimal reproducing code, so my question is only:

• What is this cast_bits operation? Is this operation not differentiable? Where does it get used (in some frequently used functions?)
• What could be the cause of this issue?
• I found something in auto_diff.cpp: TI_NOT_IMPLEMENTED;. Is this where the exception got thrown?

[Enigmatisms]

And why there will be things like the following code. Note that ray intersection has nothing to do with gradient backpropagation, I just put a dummy intersection function here (you can see color is determinstically computed, k_d has gradients). So the very existence of ray_intersection should never have affected the whole computation even though the function render is inside the Tape scope:

def render(self):
    for i in field:
        it = self.ray_intersection(ray_d, ray_o)
        color[i, j] = self.brdf[0].k_d

When I comment ray_intersection function out, I have a working (though it is useless, but at least the program is running) one but without commenting it out, I will have this error:

[E 10/09/23 16:32:15.261 10576] [auto_diff.cpp:taichi::lang::ADTransform::visit@647] Not supported.

Which literally just said nothing! I don't know what is not supported here, no other information is given and I just can't figure out why a seemingly unrelated function could affect the gradient computation, which doesn't make sense at all! Is the AD really useful in Taichi?

[jim19930609]

Hi Enigmatisms, I do apologize for the confusing error messages from Taichi.

1. The "cast_bit" operation you observed refers to an Unary operation similar to "reinterpret_cast" (in C++). Have you ever called: isinf, isnan, or bit_cast in Taichi Kernel?

2. The way AD works is that it constructs a "backward kernel" to compute for gradients with reference to the "forward kernels" (the kernel appearing under the Tape), this is similar to the concept of backpropagation graph if you're familiar with Torch/Tensorflow.

Notice that we did fixed a bunch of bugs with AD on the master's branch. The easy way to try out might be to install Taichi nightly with: pip install -i https://pypi.taichi.graphics/simple/ taichi-nightly and see if it works with latest Taichi.

[Enigmatisms]

Thanks, I will check it out. By the way, when will be version 1.7.0 get released? It's been a long time since version 1.6.0.