Weird behavior when installing from source

[amurthy1]

I know the official instructions say to download the stable version using pip install crfm-helm

But I've tried cloning and installing from source to allow file editing using pip install -e . from the cloned repo.

When installing dependent packages, it seems unable to find a valid version of boto3 and goes on searching all possible versions for a solution. Issue might be unrelated to this repo but thought I would flag for awareness. At the beginning it prints

INFO: pip is looking at multiple versions of boto3 to determine which version is compatible with other requirements. This coul
d take a while.

Then it goes on for 1000s of lines trying to find a compatible version:

Collecting boto3
  Downloading boto3-1.23.6-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 44.2 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.6
  Downloading botocore-1.26.6-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 70.8 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.5-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 32.6 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.5
  Downloading botocore-1.26.5-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 70.4 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.4-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 39.2 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.4
  Downloading botocore-1.26.4-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 55.8 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.3-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 44.1 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.3
  Downloading botocore-1.26.3-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 70.7 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.2-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 41.4 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.2
  Downloading botocore-1.26.2-py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 70.5 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.1-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 41.7 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.1
  Downloading botocore-1.26.1-py3-none-any.whl (8.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.7/8.7 MB 69.3 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.23.0-py3-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 42.8 MB/s eta 0:00:00
Collecting botocore<1.27.0,>=1.26.0
  Downloading botocore-1.26.0-py3-none-any.whl (8.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.7/8.7 MB 53.0 MB/s eta 0:00:00
Collecting boto3
  Downloading boto3-1.22.13-py3-none-any.whl (132 kB)

[amurthy1]

Looks like this is a common issue in pip, resolved this by manually installing boto3 before installing helm.

Root cause seems to be coming from here

[yifanmai]

When installing from source, you can do pip install -r requirements-freeze.txt. This will install the exact package versions that the GitHub Actions CI uses for installing from source and testing. This should allow you to avoid the pip dependency resolution issues.

[JosselinSomervilleRoberts]

Did this fix your issue?

[yifanmai]

I'm planning to make Boto an optional dependency as part of #1327, which should fix this issue.

[yifanmai]

boto3 is now an optional dependency.