Don't subtype AbstractQ <: AbstractMatrix

[dkarrasch]

This is a technically breaking experiment. I believe there is very good reason to not subtype AbstractQ as AbstractMatrix. The main reason is... it isn't! Q's are very different from container-like matrices, they are more like container-backed, function-based linear operators. I believe we've had avoidable issues in the past due to automatically allowing broadcast and indexing, see, for instance, JuliaLang/LinearAlgebra.jl#800, and method ambiguities, see the deletions in special.jl in this PR. While this may seem weird to some people, let me point out that we have nice precedence for this: AbstractRotation and Factorization, which both behave like matrices in many ways, but also don't subtype AbstractMatrix! Here, I went even one step further and introduced a minimal AdjointQ <: AbstractQ type to avoid making Q' an AbstractMatrix again. In fact, I believe the same would be reasonable for (R::AbstractRotation)', i.e., to introduce a minimal AdjointRotation type to get these things out of the matrix dispatch (see the disambiguation methods at the end of givens.jl!).

I'm not sure such a change in the type system is acceptable at all, but to the very least, we should be able to detect very bad usage of AbstractQ objects in a AbstractMatrix context (especially like in broadcasting) via a nanosoldier run, but I don't believe this will be widespread, more like by accident.

[dkarrasch]

@nanosoldier runtests(ALL, vs = ":master")

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

[dkarrasch]

@nanosoldier runtests(["AKNS", "ASE", "AbstractAlgebra", "ArrayLayouts", "BasicInterpolators", "BifurcationInference", "BosonSampling", "BugReporting", "COPT", "CairoMakie", "ControlSystems", "CountdownNumbers", "CrystalInfoFramework", "DataDeps", "DifferentiableTrajectoryOptimization", "DiffinDiffsBase", "DrelTools", "FastMarching", "FileIO", "FilesystemDatastructures", "FiniteDifferences", "FlameGraphs", "Folds", "FundamentalsNumericalComputation", "GeneNetworkAPI", "GenericLinearAlgebra", "GeoParquet", "GeometricIntegrators", "GraphMLDatasets", "GraphNeuralNetworks", "IncrementalPruning", "InfiniteArrays", "InformationGeometry", "Inpaintings", "InteractBase", "IntervalLinearAlgebra", "IscaTools", "JOLI", "JSONLines", "KernelEstimator", "KissMCMC", "KrylovMethods", "LIBSVM", "Lehmann", "LineSearches", "LinearRationalExpectations", "LoggingExtras", "LowRankArithmetic", "Lux", "MPSGE", "ManifoldLearning", "Manopt", "MatrixMarket", "Metalhead", "MultiscaleGraphSignalTransforms", "MultivariatePolynomials", "NLSolversBase", "NOMAD", "Nemo", "NiceNumbers", "NonconvexMultistart", "Optim", "OptimKit", "OptimTestProblems", "OptimizationAlgorithms", "Oscar", "ParameterSpacePartitions", "Pathfinder", "Pfam", "Pidfile", "PolaronPathIntegrals", "Polynomials", "PoreMatMod", "ProbNumDiffEq", "Probably", "ProfileView", "ProxSDP", "Pseudospectra", "QXTns", "QuadraticToBinary", "QuantumTomography", "RandomQuantum", "ReinforcementLearningExperiments", "Relief", "Retriever", "RetroCap", "Ripserer", "RungeKutta", "SDPSymmetryReduction", "SemiseparableMatrices", "SparseIR", "StateSpaceEcon", "StochasticPrograms", "StochasticRounding", "StrBase", "SuiteSparseMatrixCollection", "SunAsAStar", "SymbolicRegression", "SymbolicWedderburn", "Symbolics", "TopoPlots", "TuringGLM", "YaoBase", "YaoBlocks"], vs = ":master")

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

[dkarrasch]

This is currently failing only in SparseArrays.jl and should be easily fixable (rename Adjoint{..., <:AbstractQ} to AdjointQ{...,<:AbstractQ}). Otherwise there are only very few packages that require adjustments:

• ArrayLayouts.jl
• BandedMatrices.jl: looks like a bit more work
• GenericLinearAlgebra.jl
• packages like JOLI.jl and LinearMaps.jl, perhaps also LinearOperators.jl need to accept AbstractQs in their matrix-based operator types, should be straightforward
• SparseArrays.jl

Otherwise, this has already lead to the detection of "suboptimal" performance usage, see the list of PRs above.

Nanosoldier doesn't run GPU packages, so we may have missed breakage in that part of the ecosystem. @maleadt Would you help me detect packages that would require adjustments, or ping somebody?

Given that breakage appears to be very limited, I think this PR has some chance.

@ViralBShah @KristofferC How does it work? Is SparseArrays.jl still tied to some Julia version, or would earlier Julia versions potentially pick up (breaking) changes like this? I think they wouldn't, because older SparseArrays.jl "versions" are baked into the sysimage, right? If SparseArrays is tied to Julia, controlling the breakage would be quite easy via a VERSION-dependent branching like this

@static if VERSION > v"1.9....DEV..."
    AdjointQtype = AdjointQ
else
    AdjointQtype = Adjoint
end

and then replace the current Adjoint in method signatures by AdjointQtype?

@nanosoldier runtests(ALL, vs = ":master")

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

[andreasnoack]

This is probably the right thing to do so great that you go through all the work required to make this possible. Part of me thinks it a bit sad that we can't use the matrix abstraction for something like this but, as you point out, AbstractMatrix effectively assumes fast scalar indexing. A natural continuation would be to drop the subtyping for SparseMatrixCSC since it's pretty similar but the costs are not as extreme as for the Qs so the trade-off balance is different.

[dkarrasch]

@nanosoldier runtests(["YaoBlocks", "JOLI", "HarmonicBalance", "StochasticRounding", "ReachabilityAnalysis", "IntervalLinearAlgebra", "Pathfinder", "MultivariateStats", "Pajarito", "NumericalRepresentationTheory", "YaoBase", "DataAssimilationBenchmarks", "SBMLToolkit"])

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected.
A full report can be found here.

[dkarrasch]

All green, let's go!

[andreasnoack]

Small comment related to the git history and also the ability to review: IMO it's generally best to split refactoring and functional changes into separate PRs. It's hard to review a diff when functions are also moved to new files.

[dkarrasch]

Thanks for the feedback @andreasnoack. I should have started by first collecting all the existing methods in a new file, and then change the code.

[maleadt]

I just noticed that this broke CUDA.jl (and probably other GPU back-ends too). Specifically, our CUSOLVER wrappers testing *(Adjoint{Float32, CuMatrix{Float32}},QRPackedQ{Float32, CuMatrix{Float32}, CuVector{Float32}}), which used to dispatch to

julia/stdlib/LinearAlgebra/src/qr.jl

Lines 839 to 844 in 72aec42

	function *(adjA::Adjoint{<:Any,<:StridedVecOrMat}, Q::AbstractQ)
	A = adjA.parent
	TAQ = promote_type(eltype(A), eltype(Q))
	Ac = similar(A, TAQ, (size(A, 2), size(A, 1)))
	adjoint!(Ac, A)
	return rmul!(Ac, convert(AbstractMatrix{TAQ}, Q))

now end up in

julia/stdlib/LinearAlgebra/src/abstractq.jl

Lines 140 to 143 in ea5c9cb

	function (*)(A::AbstractMatrix, Q::AbstractQ)
	T = promote_type(eltype(A), eltype(Q))
	return rmul!(copy_similar(A, T), convert(AbstractQ{T}, Q))
	end

where the call to convert loses our GPU array data because of

julia/stdlib/LinearAlgebra/src/abstractq.jl

Line 230 in ea5c9cb

QRPackedQ{T}(Q::QRPackedQ) where {T} = QRPackedQ(convert(AbstractMatrix{T}, Q.factors), convert(Vector{T}, Q.τ))

Notice the hard-coded Vector, resulting in:

julia> Q
15×15 LinearAlgebra.QRPackedQ{Float32, CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}

julia> LinearAlgebra.QRPackedQ{T}(Q)
15×15 LinearAlgebra.QRPackedQ{Float32, CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, Vector{Float32}}

[dkarrasch]

What was the definition of convert(AbstractMatrix{TAQ}, Q) before? As far as I can tell, the two * methods are the same: for A::Adjoint the copy_similar(A, T) expands to the similar(...); adjoint!(...) code, so we need to catch the convert(AbstractQ, Q) call and make it do the same as the convert(AbstractMatrix, Q) call for Q::QRPackedQ{<:Any,<:CuArray}.

[dkarrasch]

Oh, or would it already help to convert Q.τ to AbstractVector{T}?

[maleadt]

What was the definition of convert(AbstractMatrix{TAQ}, Q) before?

It used to fall back to convert(::Type{T}, a::T) where {T<:AbstractArray} = a, so it was a no-op whereas now it does an actual conversion.

I'd also be happy to add a no-op definition for convert(AbstractQ{T}, Q::QRPackedQ{Float32, <:CuArray}), if you think that is the right thing to do.

[maleadt]

Yeah, that works:

julia> typeof(convert(LinearAlgebra.AbstractQ{T}, Q))
LinearAlgebra.QRPackedQ{Float32, CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, Vector{Float32}}

julia> LinearAlgebra.QRPackedQ{T}(Q::LinearAlgebra.QRPackedQ) where {T} = LinearAlgebra.QRPackedQ(convert(AbstractMatrix{T}, Q.factors), convert(AbstractVector{T}, Q.τ))

julia> typeof(convert(LinearAlgebra.AbstractQ{T}, Q))
LinearAlgebra.QRPackedQ{Float32, CuArray{Float32, 2, CUDA.Mem.DeviceBuffer}, CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}

[dkarrasch]

And for equal eltypes this will be a no-op as well, right? While you're at it, could you please tell me which convert method gets called? Maybe I need to review the dispatch, because I'd want the above method to be used (for equal eltypes).

[dkarrasch]

Nevermind, I found it.