Break circular reference to exc_info when exception is thrown

[homm]

This fixes #1872

I also updated tested script, it is much easier to find circular references:

https://gist.github.com/homm/c20a7dc74f78d393f6ffac537532fd93

Without fix:

$ ab -c 100 -n 10000 http://localhost:8888/dummy/
Requests per second:    1265.13 [#/sec] (mean)
Percentage of the requests served within a certain time (ms)
  50%     80
  66%     81
  75%     82
  80%     83
  90%     88
  95%    102
  98%    106
  99%    111
 100%    197 (longest request)

$ time curl http://localhost:8888/collect/
Collected: 690000
real	0m0.477s

With fix:

$ ab -c 100 -n 10000 http://localhost:8888/dummy/
Requests per second:    1346.77 [#/sec] (mean)
Percentage of the requests served within a certain time (ms)
  50%     76
  66%     77
  75%     78
  80%     79
  90%     82
  95%     88
  98%     91
  99%     92
 100%     94 (longest request)

$ time curl http://localhost:8888/collect/
Collected: 0
real	0m0.020s

[homm]

There are a lot more circular references in python 3. I'm not sure why and how to fix it simpler.

[bdarnell]

Thanks, this is awesome. I'm going to take your testing script and check it in to the repo as well so I'll be able to find it in the future.

[bdarnell]

Why did this only change in the python 3 version? Doesn't the py2 version have the same circular reference?

[ndvbd]

@bdarnell Any answer to your question? I'm getting memory leaks (python 2.7)

[bdarnell]

I put my best guess in #1936 (comment). Please do not post comments on long-closed issues. Normally I'd say you should open a new issue with details about your situation, but since tornado's support for python 2.7 ended in 2019, I'm afraid you're on your own now.

[ndvbd]

I tried adding the finally statement - didn't work. I tried upgrading python 2.7.12->2.7.17 - didn't work. Are the memory leak issues solved in python3 for Tornado?
Anyhow I see a leak with 2.7.18+5.1.1, when changing tornado 5.1.1->4.5.3 - problem is solved.

[bdarnell]

BTW, my guess about why this is more complicated on python 3 is the fact that raising an exception in an except or finally block in python 3 preserves the previous exception's traceback too. We might be able to solve it by adding from None to certain raise statements going back and trying to do that in a different way (I tried reverting your py3 commit and adding from None to raise_exc_info, but that wasn't enough).