Converting model fp32 to fp16 with auto_mixed_precision_model_path from gets NaN

[taoisu]

Hi, I was trying to convert a Bert-like model from fp32 to fp16 using the auto_mixed_precision_model_path script, I have code like below:

import fire
import onnx

import numpy as np

from onnxconverter_common.auto_mixed_precision_model_path import (
    auto_convert_mixed_precision_model_path,
)


def get_feed_tensor(name: str, max_seq_len: int):
    if name in (
        'token_type_ids',
        'segment_ids',
        'position_ids',
    ):
        return np.zeros((1, max_seq_len), dtype=np.int64)
    elif name in (
        'input_ids',
    ):
        return np.random.randint(0, 1000, (1, max_seq_len), dtype=np.int64)
    elif name in (
        'attention_mask',
    ):
        return np.ones((1, max_seq_len), dtype=np.int64)
    elif name in (
        'bbox',
    ):
        ele = np.array([0, 0, 1, 1], dtype=np.int64)
        return np.resize(ele, (1, max_seq_len, 4))
    else:
        raise NotImplementedError()


def main(
    src_model_path: str,
    tgt_model_path: str,
    max_seq_len: int = 1024,
):
    input_feed = {}
    model = onnx.load(src_model_path)
    for input in model.graph.input:
        input_feed.update({ input.name: get_feed_tensor(input.name, max_seq_len) })

    def validate_fn(res1, res2):
        for r1, r2 in zip(res1, res2):
            if not np.allclose(r1, r2, rtol=1e-2, atol=1e-2):
                return False
        return True

    auto_convert_mixed_precision_model_path(
        src_model_path,
        input_feed,
        tgt_model_path,
        provider=['CUDAExecutionProvider'],
        customized_validate_func=validate_fn,
        keep_io_types=True,
        verbose=False,
    )


if __name__ == '__main__':
    fire.Fire(main)

During conversion, I noticed some warnings about truncation, and then the validate_fn fails because res2 is all NaN, I have a screenshot showing this:

Do you have any idea how I can fix this?

[kopyl]

Same issue. Locally works fine, but in Docker – not =(

Going to try to convert with keep_io_types=True. If this won't help, i will try changing the min_positive_val and/or max_finite_val...