Add Texture, Sampling Hardware Accerlation and Mutability hints

[ifsheldon]

Concisely describe the proposed feature
Please consider design APIs for accessing texture memories and utilizing hardware-accelerated samplers.

Describe the solution you'd like (if any)
As far as I know, backends like CUDA and OpenGL have dedicated texture memory handling functions and hardware acceleration for texture fetch and sampling. But now, samplings in taichi_glsl is only soft implementations, I think. So, it will be great if you can add APIs that directly interact with the underlying hardware accelerated backend APIs.

Another related feature request will be hints for immutable data. I remember that in CUDA, there're some nuanced ways to hint cuda compiler that a block of data will not be modified in runtime and thus it can safely put the data in higher read cache hierarchy to speed up. I think you, very likely, have implemented some inferences on what data are mutated and what are not and based on that you have done some read optimizations, but it will be also great if you can give us a way to mark data's mutability to hint for optimizations.

[maajor]

+1

Please see this perftest for different type of buffer comparison.

Seems most buffer differ not much, but bilinear interpolation on Texture2D do make a difference. See the benchmark that nearest sample, bilinear sample and load on Texture2D take similar time.

As for mutability, DirectX is the same, you can declare with RW or read-only for example, but I think it need more profiling to tell if there is any difference.

[k-ye]

FYI, we are tracking this work in #5109

[ifsheldon]

Thanks for the update!

How about mutability hints? Do you have any plans of adding them? Mutability hints may be beneficial for both performance and debugging.

[k-ye]

I think mutability hints and texture support are two different things, and #5109 is talking about the later :-) But yeah, we could leverage constant memory to pass read-only data. From the mutability perspective, we only intend to support texture sampling (read-only operations) as a start, if that's what you are asking...

[ifsheldon]

From the mutability perspective, we only intend to support texture sampling (read-only operations) as a start, if that's what you are asking...

Yeah, but not only texture sampling. I mean, generally, if data is asserted to be read-only, some helpful performance optimization and checks can be (more easily) implemented. For example, I remember there're __ldg intrinsics and simple constant pointer hints for CUDA users to hint the NVCC that some data are read-only and accessed with only read operations. Probably you can let Taichi users to hint or assert that a field that is passed to a kernel is read-only so then any write operations to the read-only field is forbidden. This can help users to prevent some read-write bugs and help you Taichi developers to optimize the performance (say, loading read-only data from a field using __ldg when CUDA backend is used, or use constant memory if applicable).