problems releasing sourmash v4.2.3

[ctb]

in releasing v4.2.3 (#1761) using https://sourmash.readthedocs.io/en/latest/release.html, I ran into a problem with the first testenv (testenv1) where the pip install -r requirements.txt failed. From the error text below, it looks like it is having some trouble finding libpthread.

    gcc -pthread -shared -B /home/ctbrown/miniconda3/envs/sourmash-rc/compiler_compat -L/home/ctbrown/miniconda3/envs/sourmash-rc/lib -Wl,-rpath=/home/ctbrown/miniconda3/envs/sourmash-rc/lib -Wl,--no-as-needed -Wl,--sysroot=/ -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/home/ctbrown/miniconda3/envs/sourmash-rc/lib -Wl,-rpath-link,/home/ctbrown/miniconda3/envs/sourmash-rc/lib -L/home/ctbrown/miniconda3/envs/sourmash-rc/lib -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/ctbrown/miniconda3/envs/sourmash-rc/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/ctbrown/miniconda3/envs/sourmash-rc/include build/temp.linux-x86_64-3.8/build/temp.linux-x86_64-3.8/empty.o -o /tmp/tmp.EuG8SHPkQC/testenv1/sourmash/src/sourmash/_lowlevel__lib.so
    /home/ctbrown/miniconda3/envs/sourmash-rc/compiler_compat/ld: cannot find /lib64/libpthread.so.0
    /home/ctbrown/miniconda3/envs/sourmash-rc/compiler_compat/ld: cannot find /usr/lib64/libpthread_nonshared.a
    collect2: error: ld returned 1 exit status
    error: command '/home/ctbrown/miniconda3/envs/sourmash-rc/bin/gcc' failed with exit code 1

This looks an awful lot like pandas-dev/pandas#40706 (comment) and ContinuumIO/anaconda-issues#7363, where 'sysroot' is being incorrectly specified as /.

My current theory is that something is wonky with my sourmash-rc conda environment, which was created per our docs like so:

conda create -y -n sourmash-rc python=3.8 pip cxx-compiler make twine tox tox-conda setuptools_scm

I'm trying again with a python 3.9.environment.

UPDATE: same error as in #1689

[ctb]

I'm a little curious why we're creating a conda environment that has pip and setuptools_scm in it, and then running pip install -U setuptools pip below? It installs the latest version on of pip, which is good, I guess.

...and we do it in each virtualenv, too. Interesting.

Also, the text:

1. The below should be done in a clean checkout:

is present in the release procedure - should probably be changed to "Create a clean checkout in a temporary directory:".

[ctb]

note to self: in step 1, with python 3.9, make test failed when it ran tox -e py38. I'm trying with tox -e py39.

[ctb]

(maybe we need conda-tox?)

[ctb]

still running into problem - looks same as conda/conda#9959 -

% make test                                                                        
tox -e py39
.package create: /tmp/tmp.dpgO8Bzf82/testenv1/sourmash/.tox/.package
.package installdeps: setuptools >= 48, setuptools_scm[toml] >= 4, <6, setuptools_scm_git_archive, milksnake, wheel >= 0.29.0
ERROR: invocation failed (exit code 2), logfile: /tmp/tmp.dpgO8Bzf82/testenv1/sourmash/.tox/.package/log/.package-1.log
================================== log start ===================================
/tmp/tmpozu318c2: 5: /home/ctbrown/miniconda3/envs/sourmash-rc3/etc/conda/deactivate.d/deactivate-gcc_linux-64.sh: Syntax error: "(" unexpected

[ctb]

Can't tell if this is a farm-specific problem, or a conda problem, or what. Trying it on my laptop now.

[ctb]

It's mostly working on my laptop, except that we continue to have problems around testenv4 - installation from the test pypi server. There's something about the setuptools requirement that causes it to break; see error message below. This is particularly weird since the setuptools installed version is 60 already... wat? This is probably because setuptools isn't on the test server, but I don't know why it's not using the installed version.

We also have the problem that when pip fails to install the latest version, it cycles through all of the available versions. So we should change the command to be specific to the right version:

python -m pip install -i https://test.pypi.org/simple --pre sourmash==v${new_version}${rc}

not sure if #1360 fixes would help here but wanted to at least note those issues, too ;).

error messages and info

  ERROR: Command errored out with exit status 1:
   command: /private/var/folders/y6/s1r3q87933s6gjdrtmts1npc0000gq/T/tmp.Wm3YXQlE/testenv4/bin/python /private/var/folders/y6/s1r3q87933s6gjdrtmts1npc0000gq/T/pip-standalone-pip-puhhwrw2/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /private/var/folders/y6/s1r3q87933s6gjdrtmts1npc0000gq/T/pip-build-env-6qqpghye/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://test.pypi.org/simple --pre -- 'setuptools >= 48' 'setuptools_scm[toml] >= 4, <6' setuptools_scm_git_archive milksnake 'wheel >= 0.29.0'
       cwd: None
  Complete output (3 lines):
  Looking in indexes: https://test.pypi.org/simple
  ERROR: Could not find a version that satisfies the requirement setuptools>=48 (from versions: none)
  ERROR: No matching distribution found for setuptools>=48

pip shows that setuptools is at version 60!?

% pip show setuptools
Name: setuptools
Version: 60.0.0
Summary: Easily download, build, install, upgrade, and uninstall Python packages
Home-page: https://github.com/pypa/setuptools
Author: Python Packaging Authority
Author-email: [email protected]
License: UNKNOWN
Location: /private/var/folders/y6/s1r3q87933s6gjdrtmts1npc0000gq/T/tmp.Wm3YXQlE/testenv4/lib/python3.8/site-packages
Requires: 
Required-by: