docker base image 11-jre-hotspot-bionic -> 11-jre-hotspot-focal

[maxhbr]

This bumps the base image from adoptopenjdk:11-jre-hotspot-bionic to adoptopenjdk:11-jre-hotspot-focal. With just minor version changes.

The only complication is potentially, that the package python-pip is no longer available.

[sschuberth]

The only complication is potentially, that the package python-pip is no longer available.

So if only pip from Python 3 is available that could block the analysis of Python 2 projects, which would be an issue. Would you mind checking whether Python 2 projects can be analyzed with pip from Python 3?

[maxhbr]

FYI: query for Python2-only pip packages: https://pypi.org/search/?q=&o=&c=Programming+Language+%3A%3A+Python+%3A%3A+2&c=Programming+Language+%3A%3A+Python+%3A%3A+2+%3A%3A+Only

[maxhbr]

Using latest master, the analysis of python2 projects already fails:

$ git clone https://github.com/bayesian-trajectory-replay/bayesian_trajectory_replay
$ docker run -v "$(pwd)/bayesian_trajectory_replay:/project"  ort --info analyze -f JSON -i /project -o /project/ort/analyzer
[...]
14:31:09.639 [DefaultDispatcher-worker-3] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Resolving PIP dependencies for 'setup.py' failed with: IOException: Running '/tmp/ort-Pip-project-virtualenv5365724328195103716/bin/pip --trusted-host pypi.org --trusted-host pypi.python.org install --no-warn-conflicts --prefer-binary .' in '/project' failed with exit code 1:
  Failed building wheel for bayesian-trajectory-replay
Command "/tmp/ort-Pip-project-virtualenv5365724328195103716/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-req-build-0nvfj_q6/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-41ad7gsy/install-record.txt --single-version-externally-managed --compile --install-headers /tmp/ort-Pip-project-virtualenv5365724328195103716/include/site/python3.6/bayesian-trajectory-replay" failed with error code 1 in /tmp/pip-req-build-0nvfj_q6/
You are using pip version 18.1, however version 21.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[...]

python3 is already used to run and fails to build. The resulting analyzer output: analyzer-result.json.txt

(see: #4901)

[maxhbr]

Just tried the example from #4901 (osha.oira) with the changes here, and it successfully built the dependency tree:

 153   │       "dependency_graphs" : {
 154   │         "PIP" : {
 155   │           "packages" : [ "PyPI::setuptools:40.0.0", "PyPI::zc-buildout:2.12.1" ],
 156   │           "scopes" : {
 157   │             ":osha.oira-requirements:7.0.8.dev0:install" : [ {
 158   │               "root" : 1
 159   │             } ]
 160   │           },
 161   │           "nodes" : [ {
 162   │             "pkg" : 1
 163   │           }, { } ],
 164   │           "edges" : [ {
 165   │             "from" : 0,
 166   │             "to" : 1
 167   │           } ]
 168   │         }
 169   │       },
 170   │       "has_issues" : false
 171   │     }

full analyzer output: analyzer-result.json.txt.

So, it still has support for python2, even without python-pip.

[sschuberth]

This bumps the base image from adoptopenjdk:11-jre-hotspot-bionic to adoptopenjdk:11-jre-hotspot-focal. With just minor version changes.

Could you please add to the commit message why this is done? I.e., when you say that Bionic is too old for your use-case, please document what that use-case is in the commit message.

[sschuberth]

Also the reason for the removal of python-pip should be documented in the commit message (i.e. its unavailability in Focal), and that it has been verified that Python 2 projects can still be analyzed with python-pip3.

[maxhbr]

sure.

But also just realized that this also updates the python version to

$ python3 --version
Python 3.8.10

and the old scancode version used by default might still just support python 3.6. I will test it.

[maxhbr]

Yes, this update requires the use of a more recent scancode version, and does not work with 3.2.*.

[sschuberth]

Ok, let's bump the default version of ScanCode then finally first. I plan to do that early next year.

[sschuberth]

@maxhbr could you please rebase now that #4916 is merged? I'll then run functional tests against this Dockerfile to verify once more that nothing breaks.

[maxhbr]

It is rebased

[sschuberth]

/rebase

[sschuberth]

/rebase

[sschuberth]

@maxhbr can you please rebase manually once more as the rebase action does not seem to work on forks?

[maxhbr]

it is rebased

[sschuberth]

So, it still has support for python2, even without python-pip.

As I found out now when working on #5031, this statement is not completely true: When analyzing a Python project that only has a requirements.txt file, python3-pip implicitly assumes it's a Python 3 project, where previously it was assumed to be a Python 2 project, and only on failure Python 3 was tried.

So that's why the "Python 2" should "resolve requirements.txt dependencies correctly for example-python-flask" test currently fails when run in ORT's Docker image. I yet need to investigate how to fix this...

[sschuberth]

So, it still has support for python2, even without python-pip.

As I found out now when working on #5031, this statement is not completely true:

False alarm: The issue I saw in #5031 was not a Python 2 vs Python 3 issue, but a Python < 3.8 vs Python >= 3.8 issue. Sorry for the noise @maxhbr.