Skip to content

Commit

Permalink
Overhaul to use the new libscran libraries via pybind11 (#83)
Browse files Browse the repository at this point in the history
We no longer use our cpptypes wrapper, instead using pybind11 for convenience.
We also switch over to CMake for a more convenient compilation.

The functions in the new package are now aligned with the scrapper BioC package
for consistency. This entails the replacement of some functions (e.g., 
grouped_size_factors by compute_clrm1_factors), the renaming of others,
(e.g., score_feature_set is now score_gene_set), and some new functions
(e.g., cluster_kmeans, aggregate_across_genes).

All functions have been simplified to remove the Options classes. Instead,
people can just parametrize each step via regular function arguments.

For functions with complicated outputs, the return value is now a specialized
*Results dataclass, with an method to optionally convert it to a BiocPy class.
This is easier to document and avoids a hard dependency on BiocPy classes while
preserving interoperability when the BiocPy classes are needed.

We build the igraph static library and link to it ourselves, and use this to
enable clustering in cluster_graph(). This avoids unanticipated differences
in results due to changes in the igraph dependency. Nonetheless, our graph
objects can still be converted to igraph.Graph for interoperability.

We removed the metaprogramming for dry runs of the analyze() workflow; users
can directly inspect the source code if they are interested.

All neighbor search code has been removed in favor of a dependency on the
knncolle package, which avoids the need to recompile search code for scranpy.

The various CI workflows have been updated with the new build requirements,
and to remove support for the EOL'd Python 3.8.

Co-authored-by: Jayaram Kancherla <[email protected]>
LTLA and jkanche authored Jan 3, 2025
1 parent 6bfa315 commit 25905ce
Showing 210 changed files with 8,167 additions and 11,084 deletions.
27 changes: 10 additions & 17 deletions .github/workflows/build-docs.yml
Original file line number Diff line number Diff line change
@@ -2,45 +2,38 @@ name: Build documentation

on:
push:
branches:
- master
tags:
- "*"

jobs:
test:
name: Build docs
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
submodules: true

- name: Set up Python 3.9
uses: actions/setup-python@v2
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: 3.9
python-version: 3.11
cache: 'pip'

- name: Set up ccache
uses: hendrikmuhs/[email protected]

- name: Install Python dependencies
run: |
python -m pip install --upgrade pip setuptools
DEPENDENCIES=$(python -c 'from setuptools.config.setupcfg import read_configuration as c; a = c("setup.cfg"); print(" ".join(a["options"]["install_requires"][1:]))')
pip install ${DEPENDENCIES}
DOCDEPENDENCIES=$(python -c 'with open("docs/requirements.txt") as a: available = list(a); print(" ".join(map(lambda x : x.strip(), filter(lambda x : not x.startswith("#"), available))))')
pip install ${DOCDEPENDENCIES}
pip install cmake pybind11 numpy tox assorthead mattress>=0.1.4
# Note that doc building requires the inplace shared library.
- name: Build docs
run: |
CC="ccache gcc" python setup.py build_ext --inplace
sphinx-build --color -b html -d docs/doctrees docs docs/_build/html
python setup.py build_ext --inplace
cp build/lib*/scranpy/lib_scranpy* src/scranpy/
tox -e docs
touch ./docs/_build/html/.nojekyll
- name: GH Pages Deployment
if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/')
uses: JamesIves/[email protected]
with:
branch: gh-pages # The branch the action should deploy to.
69 changes: 69 additions & 0 deletions .github/workflows/publish-pypi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: Publish to PyPI

on:
push:
tags:
- "*"

jobs:
build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
# macos-13 is an intel runner, higher macos's are apple silicon
# At some point, maybe get this to work on windows-latest
os: [ubuntu-latest, macos-13, macos-latest]

steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Build wheels
uses: pypa/[email protected]
env:
CIBW_ARCHS_LINUX: x86_64
CIBW_PROJECT_REQUIRES_PYTHON: ">=3.9"
CIBW_SKIP: pp*

- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse/*.whl

build_sdist:
name: Build source distribution
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: true

- name: Build sdist
run: pipx run build --sdist

- uses: actions/upload-artifact@v4
with:
name: cibw-sdist
path: dist/*.tar.gz

upload_pypi:
needs: [build_wheels, build_sdist]
runs-on: ubuntu-latest
permissions:
id-token: write

steps:
- uses: actions/download-artifact@v4
with:
pattern: cibw-*
path: dist
merge-multiple: true

# This uses the trusted publisher workflow so no token is required.
- name: Publish to PyPI
if: github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/tags/')
uses: pypa/gh-action-pypi-publish@release/v1
with:
user: __token__
password: ${{ secrets.PYPI_PASSWORD }}
113 changes: 0 additions & 113 deletions .github/workflows/pypi-test.yml

This file was deleted.

34 changes: 34 additions & 0 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Test the library

on:
push:
branches:
- master
pull_request:

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]

name: Python ${{ matrix.python-version }}
steps:
- uses: actions/checkout@v4
with:
submodules: true

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"

- name: Get latest CMake
uses: lukka/get-cmake@latest

- name: Test with tox
run: |
pip install tox
tox
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -55,3 +55,7 @@ MANIFEST

# Aaron's files
playground/
installed/
extern/build-*
extern/igraph-*
extern/*.tar.gz
29 changes: 15 additions & 14 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@ exclude: '^docs/conf.py'

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
rev: v4.6.0
hooks:
- id: trailing-whitespace
- id: check-added-large-files
@@ -17,26 +17,27 @@ repos:
- id: mixed-line-ending
args: ['--fix=auto'] # replace 'auto' with 'lf' to enforce Linux/Mac line endings or 'crlf' for Windows

- repo: https://github.com/PyCQA/docformatter
rev: v1.7.5
hooks:
- id: docformatter
additional_dependencies: [tomli]
args: [--in-place, --wrap-descriptions=120, --wrap-summaries=120]
# --config, ./pyproject.toml
# - repo: https://github.com/PyCQA/docformatter
# rev: master
# hooks:
# - id: docformatter
# additional_dependencies: [tomli]
# args: [--in-place, --wrap-descriptions=120, --wrap-summaries=120]
# # --config, ./pyproject.toml

- repo: https://github.com/psf/black
rev: 23.9.1
hooks:
- id: black
language_version: python3
# - repo: https://github.com/psf/black
# rev: 24.8.0
# hooks:
# - id: black
# language_version: python3

- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: v0.0.287
rev: v0.6.8
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
- id: ruff-format

## If like to embrace black styles even in the docs:
# - repo: https://github.com/asottile/blacken-docs
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# Changelog

## Version 0.1.0 (development)
## Version 0.2.0

- Major refactor to use the new [**libscran**](https://github.com/libscran) C++ libraries.
Functions are now aligned with those in the [**scrapper**](https://bioconductor.org/packages/scrapper) package.
- Removed support for Python 3.8 (EOL).

## Version 0.1.0

- Added overlord functions for basic, multi-modal and multi-sample analyses from matrices, SummarizedExperiments and SingleCellExperiments.
2 changes: 0 additions & 2 deletions MANIFEST.in

This file was deleted.

Loading

0 comments on commit 25905ce

Please sign in to comment.