More mypy fixes/ignores #3280

Job	Run time
unittests-prototype (3.10, linux.12xlarge, cpu) / linux-job	2m 53s
unittests-prototype (3.11, linux.12xlarge, cpu) / linux-job	3m 1s
unittests-prototype (3.12, linux.12xlarge, cpu) / linux-job	3m 5s
unittests-prototype (3.8, linux.12xlarge, cpu) / linux-job	3m 6s
unittests-prototype (3.8, linux.g5.4xlarge.nvidia.gpu, cuda, 11.8) / linux-job	7m 5s
unittests-prototype (3.9, linux.12xlarge, cpu) / linux-job	2m 50s
	22m 0s

Provide feedback