Improvements for TLS with I/O threads

[uriyage]

Main thread profiling revealed significant overhead in TLS operations, even with read/write offloaded to I/O threads:

Perf results:

10.82% 8.82% valkey-server libssl.so.3 [.] SSL_pending # Called by main thread after I/O completion

10.16% 5.06% valkey-server libcrypto.so.3 [.] ERR_clear_error # Called for every event regardless of thread handling

This commit further optimizes TLS operations by moving more work from the main thread to I/O threads:

Improve TLS offloading to I/O threads with two main changes:

1. Move ERR_clear_error() calls closer to SSL operations

   • Currently, error queue is cleared for every TLS event
   • Now only clear before actual SSL function calls
   • This prevents unnecessary clearing in main thread when operations
     are handled by I/O threads

2. Optimize SSL_pending() checks

   • Add TLS_CONN_FLAG_HAS_PENDING flag to track pending data
   • Move pending check to follow read operations immediately
   • I/O thread sets flag when pending data exists
   • Main thread uses flag to update pending list

Performance improvements:
Testing setup based on https://valkey.io/blog/unlock-one-million-rps-part2/

Before:

• SET: 896,047 ops/sec
• GET: 875,794 ops/sec

After:

• SET: 985,784 ops/sec (+10% improvement)
• GET: 1,066,171 ops/sec (+22% improvement)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.68%. Comparing base (1c18c80) to head (e65c2a0).
Report is 21 commits behind head on unstable.

Additional details and impacted files

@@            Coverage Diff            @@
##           unstable    #1271   +/-   ##
=========================================
  Coverage     70.68%   70.68%           
=========================================
  Files           114      115    +1     
  Lines         63150    63158    +8     
=========================================
+ Hits          44637    44643    +6     
- Misses        18513    18515    +2     

Files with missing lines	Coverage Δ
src/tls.c	100.00% <ø> (ø)

... and 23 files with indirect coverage changes

[zuiderkwast]

Looks good to me, but I'd like @madolson to take a look too.

Impressing performance result, and this is with more lines deleted than added!

Does the change have any performance impact without I/O threads enabled?

[madolson]

Will check tomorrow, but @uriyage please fix the DCO when you get a chance.

[ranshid]

Looks good to me, but I'd like @madolson to take a look too.

Impressing performance result, and this is with more lines deleted than added!

Does the change have any performance impact without I/O threads enabled?

I would be surprised if there was any performance improvement when io-threads are not used. This will actually cause SSL_pending and ERR_clear_error to be called multiple times in some scenarios where we are reading multiple times on the read handler. BUT I think those cases are rare and AFAIK are mostly limited to cases like the connSyncReadLine.

[ranshid]

I think this will cause us to potentially keep updating the event in updateStateAfterSSLIO. Same for the connTLSSyncReadLine. Maybe you want to create an internal function which will also get the update_event indication?

[uriyage]

Thanks, I reverted this change. Instead, we will call updatePendingFlags explicitly.

[madolson]

Other than Ran's comment, this looks good to me. Nice perf results!

[uriyage]

Looks good to me, but I'd like @madolson to take a look too.

Impressing performance result, and this is with more lines deleted than added!

Does the change have any performance impact without I/O threads enabled?

As expected, there is no performance difference without I/O threads as the behavior remains the same.

I would be surprised if there was any performance improvement when io-threads are not used. This will actually cause SSL_pending and ERR_clear_error to be called multiple times in some scenarios where we are reading multiple times on the read handler. BUT I think those cases are rare and AFAIK are mostly limited to cases like the connSyncReadLine.

The only change is indeed for connSyncReadLine, but as we use it to call the read syscall one byte at a time, I believe performance is not our concern in this case.