[PROPOSAL] Remove support for EOL Python version

[dannosaur]

What/Why

What are you proposing?

In the interests of keeping the project up to date, and to be more in line with the current supported versions, I propose dropping support for Python <3.8. For reference;

• Python 3.4 EOL 18 Mar 2019
• Python 3.5 EOL 13 Sep 2020
• Python 3.6 EOL 23 Dec 2021
• Python 3.7 EOL 27 Jun 2023

Also I'd note that the elasticsearch-py package only maintains Python >= 3.6 right now, so this could be reduced to only removing support for <=3.5.

What is the developer experience going to be?

Deprecating support for EOL Python versions will allow the project to move forward and make use of newer Python features.

Are there any security considerations?

I don't think so?

Are there any breaking changes to the API

Not immediately, but I imagine there might be in the future if the project starts to maintain Python support in line with the official Python EOL schedule.

Why should it be built? Any reason not to?

I think maintaining support for (in software timelines) ancient versions of Python can hold back the project from being able to use more modern techniques, modern syntax, and take advantage of newer syntax.

What will it take to execute?

Initially nothing beyond removing the support from the documentation and setup.py. Some tests targeting (or modified) for older versions could either be removed, or the pytest.skip() calls removed to allow the tests to run.

I've removed some template questions that related to user experience, as this is just for the discussion of removing support for older Python versions that primarily affects developers.

[saimedhi]

@wbeckler, @VachaShah Please comment your thoughts :)

[dblock]

My personal 0.02c on dropping support for older versions of Python is not to do it unless there's a compelling reason (e.g. wanting to use a feature only available in a newer version, or a known security vulnerability). So is there a good reason other than it's EOL?

[dannosaur]

The line of thinking that led to me posting this was more about the async stuff, that got better from 3.6 and above. There's support in here for 3.4, 3.5, and even 2.7 - all of which end-of-life'd a long time ago. A lot of mainstream packages have also moved on, mostly only supporting 3.7 or 3.8 and above. I'll use a couple of examples, trying to focus on database engine packages;

• psycopg2: 3.6 - 3.10
• psycopg3: 3.7+
• mysqlclient: 3.8+
• elasticsearch-py: 3.6+
• redis-py: 3.7+

I think this has given them to ability to completely remove legacy code, shims that retained compatibility with older python versions, and allowed them to update syntax, usage, cleanup, etc. Personally I think its nonsensical to continue to support at the very least 2.7, but much of the world doesn't even go belong 3.6 - which just so happens to be when async started to get decent.

My $0.02 would be more about allowing the project to move forward. The last PR I did, putting tests in a place that is explicitly excluded from Pythons below 3.6 because it just doesn't work.

I suppose the alternative (though extreme) would be to fork this project into an opensearch-py-async package that is fully async, and only supports 3.6+, allowing this project to stay where it is and support as many python versions as humanly possible.

[wbeckler]

Maybe a compromise would be to wait until we have a few awkward shims, blocked upgrades and skipped tests that are noticeably holding things back, and then we make the leap? As in, let's remove support after it's noticeably hurting progress, rather than in anticipation of the need. Just a thought.

[dblock]

@dannosaur I'm actually with you. Async support is a big deal, so I vote in favor of removing everything < 3.6 and making that not optional.

[mfulgo]

FWIW, I started to wonder about supported versions (and found this proposal) while looking at what it'd take to contribute type hints to the project to resolve #402. I'm hesitant to invest the time going down that path if it will require extra effort to support EOL python versions.

[wbeckler]

I think these arguments are a convincing reason to move forward on the support drop. Now the question is, do we go with 3.6+, and is this a good time to do a major version bump on the client?

[mfulgo]

Since it requires a major version bump anyway, it seems to me that it'd be worth going straight to 3.8+ to drop support for everything that's already EOL. 🤷

[Djcarrillo6]

@saimedhi as per issue #532, I'm opened a PR that will remove end of life python version 3.5 because of security vulnerability.

[dblock]

Is it time to remove 2.x (we still test with 2.7)?
https://www.python.org/doc/sunset-python-2/

[saimedhi]

Is it time to remove 2.x (we still test with 2.7)? https://www.python.org/doc/sunset-python-2/

@Djcarrillo6 Would you like to work on removing 2.7?

[dblock]

@saimedhi I got it.

[dblock]

EOL versions are a moving target. In #548 we removed Python 2.x support. I am going to leave Python 3.7 support in, and close this.

[saimedhi]

Suggesting removal of Python versions below 3.8 for enhanced security, as both python 3.6 and 3.7 are no longer supported. Upgrading aiohttp to 3.9.0 necessitates Python 3.8 or higher. #622

[saimedhi]

Resolving all listed security vulnerabilities necessitates dropping support for EOL Python versions 3.6 and 3.7. Your thoughts on this approach would be greatly appreciated.

[dblock]

If the versions are EOL and there are security vulnerabilities to fix, we can drop them.