Remaining python2 in CMSSW

[davidlange6]

Now that #7019 landed, I wanted to make a summary of what's left in the python2 stack of cmssw and why. These are the remaining pip packages.

backports-functools_lru_cache==1.6.1
Needed by wcwidth (others are fake)

configparser==4.0.2;python_version<‘3.0'
Needed by cmsRun tasks

cython==0.29.22
Needed by scipy (not py2-numpy? Strange), rivet, yoda, professor2, gosam, lhapdf

enum34==1.1.10;python_version<‘3.0'
Needed by cmsRun tasks

future==0.18.2
Needed by cmsRun tasks

numpy==1.16.6 ; python_version<‘3.0’
Quite a few tools in cmssw as well as scipy

prettytable==1.0.1
./CondCore/Utilities/scripts/conddb:from prettytable import PrettyTable
./DQMServices/Components/scripts/dqmiodumpindices.py:from prettytable import PrettyTable

pycurl==7.43.0.5
Needed by numerous tools in cmssw

setuptools-scm==3.5.0
Needed by functools_lru_cache

six==1.15.0
Needed by cmsRun

sqlalchemy==1.3.11
A number fo cond* tools

wcwidth==0.2.5
Needed by prettytable

wheel==0.33.6
Needed by setuptools-scm

boost-python is python2

Added back:
pyyaml
Needed by conditions tools that rely on boost-python

Recently removed
py2-scipy

[cmsbuild]

A new Issue was created by @davidlange6 David Lange.

@Dr15Jones, @dpiparo, @silviodonato, @smuzaffar, @makortel, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[davidlange6]

believe most of these tools has to remain until the framework + scripts switch over (eg, a big bang to finish the migration) Exceptions appear to be

• prettytable (which allows also wcwidth functools_lru_cache, wheel, setuptools-scm)
• scipy

[davidlange6]

PR for scipy - cms-sw/cmssw#34150

[davidlange6]

PR for prettytable - cms-sw/cmssw#34151
(but I don't expect this to work)

[smuzaffar]

except future, all of these py2 packages are not part of cmssw any more

[smuzaffar]

@davidlange6 , any reason to keep python2 and py2-future in release any more?

[davidlange6]

Does crab rely on it? Otherwise, we shouldn't need it (but I have a vague memory suggesting that some external needs it - but maybe that was the py3-future package)

…

On Oct 4, 2021, at 11:38 AM, Malik Shahzad Muzaffar ***@***.***> wrote: @davidlange6 , any reason to keep python2 and py2-future in release any more? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[smuzaffar]

@davidlange6 , If I under stand correctly, crab client in crab-dev version v3.210909 uses python3 ( https://github.com/dmwm/CRABClient/tree/v3.210909/bin ) and recommendations are to use it for CMSSW >=12.0 ... right @belforte ?
None of the cmssw externals should use/depend on py2-future.

[smuzaffar]

wmagent and production system does make use of python2 and py2-future but these tools explicitly use it from release independent area /cvmfs/cms.cern.ch/COMP/slc7_amd64_gcc630/external area
FYI @amaltaro

[belforte]

correct about CRAB. We will need to put in some trick to to make a single CRAB client code switch use py2 or py3 as appropriate, but if you remove pyhon2 in CMSW_12 and make python point to python3, much easier. @ddaina is looking into making a a single CRAB Client version for all CMSSW releases (it is 95% same code already).

[smuzaffar]

Thanks @belforte for the confirmation. I am afraid making python to point to python3 might break a lot of user code. You can add something like the following in crab command

#!/bin/bash
""":"
python_cmd="python"
[ $(echo $CMSSW_VERSION | cut -d_ -f2) -gt 11 ] && python_cmd="python3"
exec ${python_cmd} $0 ${1+"$@"}
"""
#REST OF CRAB PYTHON CODE

and this should use python3 for cmssw_12 and above otherwise use python

[smuzaffar]

#7359 should drop python2 for DEVEL IBs

[smuzaffar]

python2 has been removed for 12.3.X release cycle now.