Intermittent Segfaults

[crankycyclops]

Description

I have kind of a weird situation where I have to run my unit tests a large number of times, and because of this, I uncovered an issue where maybe 5-10 times out of 1000, doctest will segfault.

After compiling my tests, I ran something like this in bash:

for i in {1..1000}; do echo "Test #$i" 2>&1 >> output; echo >> output; ./test_core 2>&1 >> output; echo >> output; done

I saved a core dump from one of the crashes, loaded it into gdb, and though I can't see any function names in the backtrace, when I list the code around where the error occurs, I get this:

(gdb) list
5977	    void registerReporterImpl(const char* name, int priority, reporterCreatorFunc c, bool isReporter) {
5978	        if(isReporter)
5979	            getReporters().insert(reporterMap::value_type(reporterMap::key_type(priority, name), c));
5980	        else
5981	            getListeners().insert(reporterMap::value_type(reporterMap::key_type(priority, name), c));
5982	    }
5983	} // namespace detail
5984	
5985	} // namespace doctest
5986	

Steps to reproduce

1. Run a binary with unit tests a large number of times (>1000)
2. Hopefully, if you're lucky, you'll trigger a segmentation fault

Extra information

Looking at my git log to see when I checked out your code so you know what I was testing, I see this:

commit d5aa2bf (HEAD -> master, tag: 2.3.8)

Compiler: g++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Operating System: Ubuntu 18.04
Doctest version: 2.3.8

[crankycyclops]

It only appears that this happens if an assertion fails. I can reliably produce at least 1 segfault if I run my tests 1000 times with a failing assertion, but if all my assertions pass, I don't get any segfaults.

[onqtam]

well... the test suite of doctest contains at least a couple dozen failing asserts and has been running on CI for a couple of years now and I've never encountered a sporadic failure like what you are describing. It could be that parts of your code corrupt some memory (could be the stack for example) and that's why it could show up as being in doctest. Also the code around registerReporterImpl is executed before even entering the main() function and I can't see a way that it could be relevant later on (in terms of callstacks). Have you tried running your code through the address sanitizer (or UB)? That could help track down any issues even in the cases when it doesn't crash.

If you can provide a small minimal example where this is reproduced and uses only doctest I'll take a look.

[crankycyclops]

Hey Viktor,

Thanks for the reply! I compiled a binary with only doctest and a failing assertion and couldn't reproduce the issue, so it looks like the problem is with my own code. Will give the address sanitizer a try and see what that uncovers.