[WIP] onednn 3.0 integration #50607
[WIP] onednn 3.0 integration #50607
Conversation
| @@ -29,11 +29,10 @@ class SoftplusOneDNNHandler | |||
| : funcs::OneDNNHandlerNoCachingT<T, dnnl::binary>(dev_ctx.GetEngine(), | |||
| dev_ctx.GetPlace()) { | |||
| dnnl::post_ops post_ops; | |||
| post_ops.append_eltwise( | |||
| 1.0f, dnnl::algorithm::eltwise_soft_relu, 0.0f, 0.0f); | |||
| post_ops.append_eltwise(dnnl::algorithm::eltwise_soft_relu, 0.0f, 0.0f); | |||
There was a problem hiding this comment.
Just a reminder:
In oneDNN v3 API, the formula of soft_relu is changed, so we may need to set the alpha to 1.f
- v2.7 soft_relu:
$d = \log_{e}(1+e^s)$ https://oneapi-src.github.io/oneDNN/v2.7/dev_guide_eltwise.html - v3 soft_relu:
$d =\frac{1}{\alpha} \log_{e}(1+e^{\alpha s})$ https://oneapi-src.github.io/oneDNN/dev_guide_eltwise.html
There was a problem hiding this comment.
@YangQun1 Thanks, will remember to modify this
onednn 3.0 quantize & dequantize
Onednn 3.0
| auto zero_points_md = dnnl::memory::desc( | ||
| {1}, dnnl::memory::data_type::s32, dnnl::memory::format_tag::x); | ||
| auto zero_points_mem = | ||
| dnnl::memory(zero_points_md, | ||
| dev_ctx.GetEngine(), | ||
| phi::funcs::to_void_cast<int32_t>(&quantization_shift)); |
There was a problem hiding this comment.
need to check with shift?
| post_operations.append_eltwise(dnnl::algorithm::eltwise_relu, 0.0f, 0.0f); | ||
| post_operations.append_eltwise( | ||
| activation_scale, dnnl::algorithm::eltwise_relu, 0.0f, 0.0f); | ||
| dnnl::algorithm::eltwise_linear, activation_scale, 0.0f); |
There was a problem hiding this comment.
Why not use DNNL_ARG_DST SCALE for activation_scale since it will be applied after activation.
| attributes.set_output_scales(mask, output_shift_scale); | ||
| attributes.set_scales_mask(DNNL_ARG_DST, mask); |
There was a problem hiding this comment.
OUTPUT_SCALE is not equal to DST SCALE
| @@ -37,37 +37,22 @@ class LayerNormOneDNNHandler | |||
| engine, cpu_place) { | |||
| const auto fwd_prop_kind = is_test ? dnnl::prop_kind::forward_inference | |||
| : dnnl::prop_kind::forward_training; | |||
|
|
|||
There was a problem hiding this comment.
A random place:
Will these oneDNN kernels in fluid/operators/mkldnn dir be migrated to phi/kernels/onednn dir? Are we doing this task?
There was a problem hiding this comment.
Hi, oneDNN kernels migration to PHI is finished. Kernels which are still in fluid are left on purpose (description: #45308)
There was a problem hiding this comment.
Got it, thanks for your explanation
|
The convolution bias scaling process may be not needed anymore, since oneDNN v3.0 move the bias add after the dequantization. (Refer to the new quantization scheme: https://github.com/oneapi-src/oneDNN/tree/rfcs/rfcs/20220201-quantization-scaling) |
Elementwise add & mul fix
Fix Elementwise div
|
onednn v3.x has been upgraded in #52463. Thanks for the initial PR:) |
PR types
PR changes
Describe