Implement predict_per_tree() in FIL

[hcho3]

No description provided.

[wphicks]

Looking great! I left some inline feedback on details, but in general it looks quite good.

The one larger thing I'd like to see us clean up is how we're handling the shared memory to global memory fallback. Rather than having custom logic at every point we need to touch that, it would be nice to encapsulate all of that in shared_memory_buffer. Specifically, I would love to see the signature for fill change to

fill(index_type element_count, T value=T{}, T* fallback_buffer=nullptr)

Then, we don't have to have any special logic for when we are or are not using the fallback. Before we launch the kernel, we allocate global memory if we detect that we don't have enough shared memory. Otherwise, we have an empty buffer. We then pass that pointer all the way through the layers without any other processing down to shared_memory_buffer. If it sees that the fill is going to fail, it fills the fallback buffer and returns that pointer. That keeps the logic for all different prediction types the same, and it encapsulates any special handling in the shared_memory_buffer object.

Other than that, I think we're looking pretty great!

[hcho3]

FYI, I renamed the enum throughout the codebase so that it's not confused with the array type.

• output_type -> infer_type or predict_type, depending on context.
• output_kind -> infer_kind

[wphicks]

Beautiful! Everything about this is great. I had one small request for where we put the output_t logic, but it should not hold up a merge unnecessarily.

I'm testing for perf regressions now, and we can merge as soon as that gets cleared. If we get that output_t change in before that's done, great, but otherwise I'm fine with merging as is.

[wphicks]

/merge