Pipenv install fails on trying to install ibm packages

[oindrillac]

Describe the bug
Pipenv install of ibm-cloud-sdk-core and ibm-watson packages return errors.

To Reproduce
Steps to reproduce the behavior:

1. Spawn a server on https://jupyterhub-opf-jupyterhub.apps.cnv.massopen.cloud/
2. Spin up a 'mailing-list-analysis-toolkit:latest' image
3. git clone [email protected]:aicoe-aiops/mailing-list-analysis-toolkit.git
4. cd mailing-list-analysis-toolkit/
5. pipenv install succeeds
6. pipenv install ibm-cloud-sdk-core fails.

with following error

[1000600000@jupyterhub-nb-oc-40bu-2eedu mailing-list-analysis-toolkit]$ git statusOn branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
[1000600000@jupyterhub-nb-oc-40bu-2eedu mailing-list-analysis-toolkit]$ pipenv install ibm-cloud-sdk-core
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing ibm-cloud-sdk-core…
Adding ibm-cloud-sdk-core to Pipfile's [packages]…
✔ Installation Succeeded
Pipfile.lock (bcb5eb) out of date, updating to (66307a)…
Locking [dev-packages] dependencies…
✔ Success!
Locking [packages] dependencies…
✘ Locking Failed!
[pipenv.exceptions.ResolutionFailure]:   File "/opt/app-root/lib64/python3.8/site-packages/pipenv/resolver.py", line 61, in resolve
[pipenv.exceptions.ResolutionFailure]:       return resolve_deps(
[pipenv.exceptions.ResolutionFailure]:   File "/opt/app-root/lib64/python3.8/site-packages/pipenv/utils.py", line 718, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree, hashes, markers_lookup, resolver = actually_resolve_deps(
[pipenv.exceptions.ResolutionFailure]:   File "/opt/app-root/lib64/python3.8/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/opt/app-root/lib64/python3.8/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure:ERROR: ERROR: Could not find a version that matches black
[pipenv.exceptions.ResolutionFailure]:       Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1, 18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0, 19.10b0, 19.10b0, 20.8b0, 20.8b1
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches black
Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1, 18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1,18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0, 19.10b0, 19.10b0, 20.8b0, 20.8b1
There are incompatible versions in the resolved dependencies.
[pipenv.exceptions.ResolutionFailure]:       return resolve_deps(
[pipenv.exceptions.ResolutionFailure]:   File "/opt/app-root/lib64/python3.8/site-packages/pipenv/utils.py", line 718, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree, hashes, markers_lookup, resolver = actually_resolve_deps(
[pipenv.exceptions.ResolutionFailure]:   File "/opt/app-root/lib64/python3.8/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/opt/app-root/lib64/python3.8/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure:ERROR: ERROR: Could not find a version that matches black
[pipenv.exceptions.ResolutionFailure]:       Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1, 18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0, 19.10b0, 19.10b0, 20.8b0, 20.8b1
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches black
Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1, 18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1,18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0, 19.10b0, 19.10b0, 20.8b0, 20.8b1

[tumido]

yes, this is caused by transitive dependency on a package pre-release see here:
thoth-station/thoth-application#816

We use pipermill which is a normally released package, which unfortunately depends on black - a package that has no release, only pre-releases for ages now. 🤷‍♂️

Once this thoth-station/thoth-application#884 is resolved and new thamos is released, we should be using thamos install instead of pipenv install here.

Right now please try running pipenv lock --pre to enable pre-releases for the lock run (it may conflict with the explicit toggle in the Pipfile though, we'll see).

[oindrillac]

Thanks @tumido
Running pipenv lock --pre
and pipenv install --pre ibm-cloud-sdk-core worked for me