Python3.7 ValueError: attempt to connect already-connected SSLSocket!

[dwintergruen]

Try to establish a producer with:

(python3.7 Mac OS high Sierra)

producer = KafkaProducer(bootstrap_servers='SERVER:9093', ssl_password="XXXXX",
ssl_cafile="/etc/kafka_clients/ca-cert", security_protocol="SSL",
ssl_certfile="/etc/kafka_clients/cert-signed",
ssl_keyfile="/etc/kafka_clients/private-key")

always get:

File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/kafka/producer/kafka.py", line 362, in init
**self.config)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/kafka/client_async.py", line 214, in init
self._bootstrap(collect_hosts(self.config['bootstrap_servers']))
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/kafka/client_async.py", line 245, in _bootstrap
if not bootstrap.connect_blocking():
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/kafka/conn.py", line 301, in connect_blocking
self.connect()
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/kafka/conn.py", line 354, in connect
ret = self._sock.connect_ex(self._sock_addr)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1146, in connect_ex
return self._real_connect(addr, True)
File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1118, in _real_connect
raise ValueError("attempt to connect already-connected SSLSocket!")
ValueError: attempt to connect already-connected SSLSocket!

[freeyoung]

I'm getting exactly the same exception. Python 3.6 & 3.5 work prefectly.

[bunnyc1986]

I am getting the same error. Python 3.7 as well.

May be related to this change: https://github.com/python/cpython/pull/5252/files#diff-c49248c7181161e24048bec5e35ba953R1108

[apoclyps]

I've encountered the same problem described above whilst upgrading to python:3.7-alpine. I've created a test case which should allow easy reproduction of the issue.

I've attempted to experiment with different kafka-python versions but found the issue described above limited to 1.4.3. 1.4.2 produces a different issue, outlined separately at the bottom of issue

[matthieugouel]

Hello,

As @bunnyc1986 pointed out, it has to do with a modification of the SSL module of CPython. I tried to remove the self._sslobj is not None of the line 1108 and it worked perfectly, but I guess the underlaying issue comes from kafka-python which may try to do a connection with the same socket multiple times.

It's a little annoying because it prevents us to upgrade python to 3.7 just for that tiny issue ...

[richtera]

Any news on this?

[jeffwidman]

A PR would be welcome from someone who can test on permutations of SSL enabled / disabled for python 2.7, 3.6, and 3.7.

[T-baby]

When can python-kafka fix this problem?

[dpkp]

We are waiting for a kind soul like yourself to develop a fix, test it with different python/ssl configurations, and submit a PR.

…

On Wed, Nov 14, 2018, 6:27 AM T-baby ***@***.*** wrote: When can python-kafka fix this problem? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1549 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAzetKthOceRiB9uNirfPvyvRrbWAj4qks5uvChFgaJpZM4VTS1l> .

[T-baby]

Come and see it every day. When can python-kafka fix this problem?

[dpkp]

kafka-python fix is one PR away! Will happily accept your improvements!

…

On Wed, Nov 21, 2018, 7:06 PM T-baby ***@***.*** wrote: Come and see it every day. When can python-kafka fix this problem? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1549 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAzetFSDxD2gOoZEOtwpcobjTejJqJADks5uxhSigaJpZM4VTS1l> .

[seanthegeek]

@dpkp Here's that PR :)

[silentben]

@dpkp Looks like both @seanthegeek and I have submitted pull requests to fix this. Anything you're looking for before merging one or the other?

[jeffwidman]

Fixed by #1669

[slyons]

This is still an issue in the current release. Any chances of a newer release?