Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lanczos Solver #2416

Closed
wants to merge 25 commits into from
Closed

Conversation

aamijar
Copy link
Contributor

@aamijar aamijar commented Aug 20, 2024

Lanczos Solver for Sparse Eigen Decomposition

We propose a new lanczos solver in raft that fixes the issues present in the previous solver raft::sparse::solver::detail::computeSmallestEigenvectors.

Specifically we address the following issues:

  1. Numerical Stability for both float32 and float64 datatypes
  2. Efficiency and Speed of Convergence

This new implementation is taken from the cupy library cupyx.scipy.sparse.linalg.eigsh where the thick-restart and full reorthogonalzation methods are used.

Additionally this PR exposes a python api for raft lanczos solver with an interface similar to scipy.sparse.linalg.eigsh and cupyx.scipy.sparse.linalg.eigsh.

from pylibraft.solver import eigsh

Copy link

copy-pr-bot bot commented Aug 20, 2024

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions bot added the cpp label Aug 20, 2024
@aamijar aamijar force-pushed the lanczos-solver-new branch from 7ec5f7f to 0f5bdcb Compare August 20, 2024 23:45
@aamijar aamijar mentioned this pull request Aug 20, 2024
@aamijar aamijar self-assigned this Aug 21, 2024
@aamijar aamijar added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Aug 21, 2024
@aamijar aamijar marked this pull request as ready for review August 21, 2024 23:24
@aamijar aamijar requested review from a team as code owners August 21, 2024 23:24
Copy link
Member

@cjnolet cjnolet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is coming along great, @aamijar. Mostly it needs some cleanup and polishing, but otherwise should be ready to merge once my comments are resolved.

cpp/include/raft/sparse/solver/detail/lanczos.cuh Outdated Show resolved Hide resolved
cpp/include/raft/sparse/solver/detail/lanczos.cuh Outdated Show resolved Hide resolved
cpp/include/raft/sparse/solver/detail/lanczos.cuh Outdated Show resolved Hide resolved
cpp/include/raft/sparse/solver/detail/lanczos.cuh Outdated Show resolved Hide resolved
cpp/src/raft_runtime/solver/lanczos_solver.cuh Outdated Show resolved Hide resolved
cpp/include/raft_runtime/solver/lanczos.hpp Outdated Show resolved Hide resolved
@aamijar aamijar force-pushed the lanczos-solver-new branch from 81475a6 to 2be393a Compare August 22, 2024 02:15
cpp/include/raft/sparse/solver/lanczos.cuh Outdated Show resolved Hide resolved
cpp/include/raft/sparse/solver/lanczos.cuh Show resolved Hide resolved
cpp/include/raft_runtime/solver/lanczos.hpp Outdated Show resolved Hide resolved
cpp/include/raft/sparse/solver/lanczos.cuh Show resolved Hide resolved
python/pylibraft/pylibraft/solver/__init__.py Outdated Show resolved Hide resolved
cpp/include/raft/sparse/solver/detail/lanczos.cuh Outdated Show resolved Hide resolved
cpp/include/raft/sparse/solver/lanczos.cuh Show resolved Hide resolved
cpp/test/sparse/solver/lanczos.cu Outdated Show resolved Hide resolved
cpp/test/sparse/solver/lanczos.cu Show resolved Hide resolved
@aamijar aamijar force-pushed the lanczos-solver-new branch from bd26c79 to 7b31108 Compare August 28, 2024 16:13
@aamijar aamijar requested a review from lowener September 9, 2024 18:21
Copy link
Contributor

@lowener lowener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can a test for the python API of Lanczos also be added?

cpp/include/raft/sparse/solver/lanczos.cuh Show resolved Hide resolved
cpp/include/raft/sparse/solver/lanczos.cuh Show resolved Hide resolved
handle,
v0_vector_const,
V_0_view,
[device_scalar = v0nrm_scalar.data_handle()] __device__(auto y) { return y / *device_scalar; });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can v0nrm and it's copy operations be skipped this way?

Suggested change
[device_scalar = v0nrm_scalar.data_handle()] __device__(auto y) { return y / *device_scalar; });
[device_scalar = output1.data_handle()] __device__(auto y) { return y / *device_scalar; });

@lowener lowener mentioned this pull request Nov 1, 2024
@lowener
Copy link
Contributor

lowener commented Nov 1, 2024

I unfortunately don't have permissions to push on the branch of this PR so I kept the commits and continued on PR #2481.

@cjnolet cjnolet closed this Nov 13, 2024
rapids-bot bot pushed a commit that referenced this pull request Nov 20, 2024
I unfortunately don't have permissions to push on @aamijar branch for the previous Lanczos solver PR (#2416) so I kept his commits and continued it here.

## Lanczos Solver for Sparse Eigen Decomposition
We propose a new lanczos solver in raft that fixes the issues present in the previous solver `raft::sparse::solver::detail::computeSmallestEigenvectors`.

Specifically we address the following issues:
1. Numerical Stability for both float32 and float64 datatypes
2. Efficiency and Speed of Convergence

This new implementation is taken from the cupy library `cupyx.scipy.sparse.linalg.eigsh` where the thick-restart and full reorthogonalzation methods are used.

Additionally this PR exposes a python api for raft lanczos solver with an interface similar to `scipy.sparse.linalg.eigsh` and `cupyx.scipy.sparse.linalg.eigsh`.

```py3
from pylibraft.solver import eigsh
```

Authors:
  - Micka (https://github.com/lowener)
  - Anupam (https://github.com/aamijar)
  - Corey J. Nolet (https://github.com/cjnolet)

Approvers:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)
  - Corey J. Nolet (https://github.com/cjnolet)

URL: #2481
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CMake cpp improvement Improvement / enhancement to an existing function non-breaking Non-breaking change python
Projects
Development

Successfully merging this pull request may close these issues.

3 participants