zio_ddt_write: Assertion `bp->blk_birth == txg' failed

[behlendorf]

The following assert is now occasionally tripped by ztest in automated testing. I don't believe this issue was hit prior to the 0.6.4 tag so it was most likely introduced by one of the many illumos patches which were merged. If someone has the time to bisect where this was introduced that would be very helpful.

52.4M completed (  53MB/s) estimated time remaining: 0hr 00min 01sec        
 106M completed (  53MB/s) estimated time remaining: 0hr 00min 00sec        ztest: ../../module/zfs/zio.c:2429: zio_ddt_write: Assertion `bp->blk_birth == txg' failed.
/usr/sbin/ztest[0x40bc9a]
/lib64/libpthread.so.0(+0xf130)[0x7f3cf69ff130]
/lib64/libc.so.6(gsignal+0x37)[0x7f3cf66645d7]
/lib64/libc.so.6(abort+0x148)[0x7f3cf6665cc8]
/lib64/libc.so.6(+0x2e546)[0x7f3cf665d546]
/lib64/libc.so.6(+0x2e5f2)[0x7f3cf665d5f2]
/lib64/libzpool.so.2(+0xf9583)[0x7f3cf7c75583]
/lib64/libzpool.so.2(zio_execute+0x113)[0x7f3cf7c6dca3]
/lib64/libzpool.so.2(+0x2e64d)[0x7f3cf7baa64d]
/lib64/libzpool.so.2(zk_thread_helper+0x1f7)[0x7f3cf7ba4927]
/lib64/libpthread.so.0(+0x7df5)[0x7f3cf69f7df5]
/lib64/libc.so.6(clone+0x6d)[0x7f3cf67251ad]
child died with signal 6
program finished with exit code 3
elapsedTime=277.230543

[kernelOfTruth]

I'm currently not in my Linux system - so no access to in-commit search

a quick skim revealed that there were however changes to bp->blk_birth just recently:

e5fd1dd
Illumos 5243 - zdb -b could be much faster

[behlendorf]

@ahrens does this look familiar? It was very likely introduced by one of the patches we recently pulled from illumos, possibly e5fd1dd although we haven't bisected it yet.

[ahrens]

Yes, I've seen this on illumos too. Not sure of the root cause.

[kernelOfTruth]

openzfs/openzfs@b39b744

https://www.illumos.org/issues/6319 assertion failed in zio_ddt_write: bp->blk_birth == txg