Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalize indices in promote_shape error messages. #40124

Closed
wants to merge 471 commits into from

Conversation

tpapp
Copy link
Contributor

@tpapp tpapp commented Mar 21, 2021

Seeing implementation like Base.OneTo in error messages may be confusing to some users (cf discussion in #39242,
discourse).

This PR turns

julia> ones(2, 3) + ones(3, 2)
ERROR: DimensionMismatch("dimensions must match: a has dims (Base.OneTo(2), Base.OneTo(3)), b has dims (Base.OneTo(3), Base.OneTo(2)), mismatch at 1")

into

julia> ones(2, 3) + ones(3, 2)
ERROR: DimensionMismatch("dimensions must match: a has axes (1:2, 1:3), b has axes (1:3, 1:2), mismatch at 1")

Fixes #40118.

@tpapp tpapp added error messages Better, more actionable error messages minor change Marginal behavior change acceptable for a minor release labels Mar 21, 2021
@Cvikli
Copy link

Cvikli commented Mar 21, 2021

But why not using dims when the programmer needs dims.

I mean, this is still confusing for me as an error message. Any programmer that used python or C++ would expect dims to be numbers instead of ranges. Also design pattern we should follow Jacob Law (https://lawsofux.com/jakobs-law/). I mean I am a CEO of a company and any time we design the product and created a new paradigm sooner or later we faced with problem that 99% of the users are getting used to the other pattern that is used on everyone else's platform. Many advisor I meet suggested the same and some of them were leader of 1000 employee big companies.

I would suggest to use this one:
ERROR: DimensionMismatch("dimensions must match: a has dims (2, 3), b has dims (3, 2), mismatch at 1")

That is why I proposed the solution on the other issue. promote_shape dimension mismatch prompt

I don't understand why do you force the other way. Can you argue why you want this and not the other one?

@tpapp
Copy link
Contributor Author

tpapp commented Mar 21, 2021

I would suggest to use this one:
ERROR: DimensionMismatch("dimensions must match: a has dims (2, 3), b has dims (3, 2), mismatch at 1")

I guess we could specialize for Base.OneTo, but I find the proposed solution consistent for custom indexing.

I don't understand why do you force the other way.

You misunderstand the process — this is a pull request, not yet merged. I am not forcing anything, comments are welcome.

@Cvikli
Copy link

Cvikli commented Mar 21, 2021

Base.OneTo is nice thing for some general version of the Array and the printing is cleaner as you described for general Arrays which is nice for advanced usage of for example: OffsetArrays where there is a scenario as I understood when it isn't a dim it is already a range.

What I would propose is in case of concrete Array types, to use a cleaner version. For example one possible way is just extend the promote_shape:

import Base: promote_shape
function promote_shape(a::Array{A}, b::Array{B}) where {A,B}
    promote_shape(size(a), size(b))
end

Which solve the whole error prompt and you get nice and clean:
ERROR: DimensionMismatch("dimensions must match: a has dims (2, 3), b has dims (3, 2), mismatch at 1")

This uses common pattern that is understandable for everyone coming from other languages.

@jishnub
Copy link
Contributor

jishnub commented Mar 21, 2021

Perhaps the error message might be altered here, as ultimately the check is for the indices of the arrays and not the sizes. Something along the lines of "axes must match: a has indices (1:2, 1:3), b has indices (1:3, 1:2), mismatch at 1" might make it clearer that ultimately it's the indices that must be identical for the operation to succeed. This also removes the possible ambiguity in the meaning of "dims".

Although consistency might be harder to obtain in general, as StaticArrays checks the sizes and ignores the indices. For example, why does this work

julia> ones(2:4,2) + SMatrix{3,2}(ones(3,2))
3×2 SArray{Tuple{3,2},Float64,2,6} with indices SOneTo(3)×SOneTo(2):
 2.0  2.0
 2.0  2.0
 2.0  2.0

while this doesn't:

julia> ones(2:4,2) + ones(3,2)
ERROR: DimensionMismatch("dimensions must match: a has dims (OffsetArrays.IdOffsetRange(2:4), OffsetArrays.IdOffsetRange(1:2)), b has dims (Base.OneTo(3), Base.OneTo(2)), mismatch at 1")

The indices are clearly different in the first case, but that doesn't seem to matter. The broader issue of indices vs. size might need some work across the ecosystem to fix.

@tpapp
Copy link
Contributor Author

tpapp commented Mar 21, 2021

@jishnub:

why does this work
ones(2:4,2) + SMatrix{3,2}(ones(3,2))

Because StaticArrays defines a method for + which ultimately dispatches to something that skips the shape check (other than sizes).

The broader issue of indices vs. size might need some work across the ecosystem to fix.

My understanding is that promote_shape is somewhat vestigial at this point, a remnant of earlier code that is mostly superseded by broadcasting (@mbauman, is this correct?). It is used only in a few places, called with Ints by zip and axes by + and - (for arrays). It is not something the ecosystem should really rely on any more.

@tpapp tpapp requested a review from kshyatt March 22, 2021 14:34
@tpapp
Copy link
Contributor Author

tpapp commented Mar 22, 2021

@jishnub, thanks for the suggestion. I went with "axes" and "size", since the two contexts that call Base.promote_shape are array operations and iteration.

@Cvikli
Copy link

Cvikli commented Mar 22, 2021

Does _has_axes = T <: AbstractUnitRange evaluate at compilation time? It look nicer if it is splitted by function type?

@tpapp
Copy link
Contributor Author

tpapp commented Mar 23, 2021

@Cvikli: yes, this information should be known at compile time. But for the function which just throws an error that should be largely irrelevant. In fact, I was considering using @nospecialize, but it should be no big deal either way.

It look nicer if it is splitted by function type?

I don't understand what you mean here.

@tpapp
Copy link
Contributor Author

tpapp commented Apr 2, 2021

Closing and opening to trigger CI.

It would be great if someone could review this. @kshyatt, since you modified this code the last time, your comments/review would be appreciated.

@tpapp tpapp closed this Apr 2, 2021
@tpapp tpapp reopened this Apr 2, 2021
@mbauman
Copy link
Member

mbauman commented Apr 2, 2021

Yeah, we don't really need to worry about performance in an error message — if anything I'd lean towards pessimizing the performance to reduce compile time.

What about further simplifying this to use the size version for OneTo axes?

(Also this isn't a minor change as it doesn't even change the error type; we're free to change printing from version to version)

@mbauman mbauman removed the minor change Marginal behavior change acceptable for a minor release label Apr 2, 2021
@tpapp
Copy link
Contributor Author

tpapp commented Apr 2, 2021

@mbauman: I am not sure why we would special-case Base.OneTo here. Consider axes Base.OneTo(4) and 1:4:
my understanding from #39242 is that they should print the same, as Base.OneTo is an implementation detail.

Also, consistently printing something that axes could return may be good practice.

PaulSoderlind and others added 12 commits April 19, 2021 05:47
This adds a `selectdim(A, 2, 3:4)` example to illustrate that several indices can be used (as this is not entirely clear in the text). Based on a suggestion in https://discourse.julialang.org/t/pick-rows-from-array-ndims-not-known-in-advance/59033/6
From JuliaLang#39432 and JuliaLang#39441, these were still using their old definition due
to method overwriting.
JeffBezanson and others added 18 commits May 27, 2021 17:46
…ght-be-fixed

Reenable some errorshow tests on FreeBSD
Jeff was a bit sceptical about this in JuliaLang#40574, but I think this
is worth doing since it seems strictly more useful than the current
behavior and we already go out of our way in cases like `x, y = y, x` to
only assign to the lhs after the rhs has been evaluated, so I think
handling `x[2], x[1] = x` in a similar way would only be consistent. As
a general rule that assignment will always happen after destructuring
does not seem much less obvious than the current behavior to me. This
might technically be slightly breaking, but I would be surprised if
people relied on the current behavior here. Nevertheless, it would
probably be good to run PkgEval if we decide to go with this change.

closes JuliaLang#40574
Co-authored-by: Jeff Bezanson <[email protected]>
Co-authored-by: Fredrik Ekre <[email protected]>
Co-authored-by: Kristoffer Carlsson <[email protected]>
This allows code to assume that `->layout` will be assigned when
analyzing a type in codegen if the type is mapped to C. The ABI code
often assumes this, and it is also just much generally easier that the
jl_struct_to_llvm code can share the jl_compute_field_offsets results.
We generally seem to prefer sentences over keyword lists (which feel a
bit too much like Javadoc to me), so this changes the documentation to
use a more free-form sentence for "see also," including an ending period
(we seem to have intuitively preferred this already, by a ratio of 5:2).
fix unsoundness in fieldtype of Tuple with non-Type params (fixes JuliaLang#39988)
Without this change, suitesparse checksum files don't get packed
properly.
* Remove SuiteSparse_wrapper and winclang patch

Fix https://github.com/JuliaLang/SuiteSparse.jl/issues/11
Fix https://github.com/JuliaLang/julia/issues/37322

* Update SuiteSparse version to 5.10.1

* Update libsuitesparse checksums for 5.10.1

* Remove more wrapper stuff
Previously the printing pass for converted data only works for
non-tabular data (like simple number literals), and it doesn't work for
`Dict` or `Array`s. Rather it leads to runtime error because we don't
pass over the same `by` keyword  argument through recursive calls and it
may not be assigned:
```julia
julia> struct MyStruct
           a::Int
       end

julia> data = Dict(:foo => MyStruct(1))
Dict{Symbol, MyStruct} with 1 entry:
  :foo => MyStruct(1)

julia> TOML.print(data; softed=true) do x
           x isa MyStruct && return Dict(:bar => x.a)
       end
ERROR: MethodError: no method matching print(::var"JuliaLang#38#39", ::Dict{Symbol, MyStruct}; softed=true)
You may have intended to import Base.print
Closest candidates are:
  print(::Union{Nothing, Function}, ::AbstractDict; sorted, by) at /Users/aviatesk/julia/julia/stdlib/TOML/src/print.jl:130 got unsupported keyword argument "softed"
  print(::Union{Nothing, Function}, ::IO, ::AbstractDict; sorted, by) at /Users/aviatesk/julia/julia/stdlib/TOML/src/print.jl:129 got unsupported keyword argument "softed"
  print(::IO, ::AbstractDict; sorted, by) at /Users/aviatesk/julia/julia/stdlib/TOML/src/print.jl:131 got unsupported keyword argument "softed"
Stacktrace:
 [1] kwerr(::NamedTuple{(:softed,), Tuple{Bool}}, ::Function, ::Function, ::Dict{Symbol, MyStruct})
   @ Base ./error.jl:163
 [2] top-level scope
   @ none:1
```

<details><summary>Originally reported by JET:</summary>
julia> using JET, Pkg

julia> @report_call Pkg.project()
═════ 3 possible errors found ═════
┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/Pkg/src/API.jl:103 Pkg.API.EnvCache()
│┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/Pkg/src/Types.jl:285 #self#(Pkg.Types.nothing)
││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/Pkg/src/Types.jl:288 Pkg.Types.read_project(project_file)
│││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/Pkg/src/project.jl:138 Pkg.Types.sprint(Pkg.Types.showerror, e)
││││┌ @ strings/io.jl:106 Base.#sprint#412(Core.tuple(Base.nothing, 0, #self#, f), args...)
│││││┌ @ strings/io.jl:112 f(Core.tuple(s), args...)
││││││┌ @ toml_parser.jl:326 Base.TOML.point_to_line(Base.getproperty(err, :str), pos, pos, io)
│││││││ for 1 of union split cases, no matching method found for call signatures (Tuple{typeof(Base.TOML.point_to_line), Nothing, Int64, Int64, IOBuffer})): Base.TOML.point_to_line(Base.getproperty(err::Base.TOML.ParserError, :str::Symbol)::Union{Nothing, String}, pos::Int64, pos::Int64, io::IOBuffer)
││││││└──────────────────────
│││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/Pkg/src/project.jl:142 Pkg.Types.Project(raw)
││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/Pkg/src/project.jl:124 Base.setproperty!(project, :targets, Pkg.Types.read_project_targets(Pkg.Types.get(raw, "targets", Pkg.Types.nothing), project))
│││││┌ @ Base.jl:35 Base.convert(Base.fieldtype(Base.typeof(x), f), v)
││││││┌ @ abstractdict.jl:523 _(x)
│││││││┌ @ dict.jl:104 Base.setindex!(h, v, k)
││││││││┌ @ dict.jl:382 Base.convert(_, v0)
│││││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/LinearAlgebra/src/factorization.jl:58 _(f)
││││││││││ no matching method found for call signature (Tuple{Type{Vector{String}}, LinearAlgebra.Factorization}): _::Type{Vector{String}}(f::LinearAlgebra.Factorization)
│││││││││└─────────────────────────────────────────────────────────────────────────────────────────────────
││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/Pkg/src/Types.jl:305 Pkg.Types.write_env_usage(manifest_file, "manifest_usage.toml")
│││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/Pkg/src/Types.jl:436 Pkg.Types.sprint(JuliaLang#35)
││││┌ @ strings/io.jl:106 Base.#sprint#412(Core.tuple(Base.nothing, 0, #self#, f), args...)
│││││┌ @ strings/io.jl:112 f(Core.tuple(s), args...)
││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/Pkg/src/Types.jl:437 TOML.print(io, Pkg.Types.Dict(Pkg.Types.=>(Core.getfield(#self#, :source_file), Base.vect(Pkg.Types.Dict(Pkg.Types.=>("time", Pkg.Types.now()))))))
│││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/TOML/src/print.jl:131 TOML.Internals.Printer.#print#16(false, TOML.Internals.Printer.identity, #self#, io, a)
││││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/TOML/src/print.jl:131 Core.kwfunc(TOML.Internals.Printer._print)(Core.apply_type(Core.NamedTuple, (:sorted, :by))(Core.tuple(sorted, by)), TOML.Internals.Printer._print, TOML.Internals.Printer.nothing, io, a)
│││││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/TOML/src/print.jl:76 #s848(_2, _3, f, io, a, Base.getindex(TOML.Internals.Printer.String))
││││││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/TOML/src/print.jl:76 TOML.Internals.Printer.#_print#11(indent, first_block, sorted, by, _3, f, io, a, ks)
│││││││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/TOML/src/print.jl:88 Core.kwfunc(TOML.Internals.Printer.printvalue)(Core.apply_type(Core.NamedTuple, (:sorted,))(Core.tuple(sorted)), TOML.Internals.Printer.printvalue, f, io, value)
││││││││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/TOML/src/print.jl:25 TOML.Internals.Printer.#printvalue#1(sorted, _3, f, io, value)
│││││││││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/TOML/src/print.jl:29 Core.kwfunc(TOML.Internals.Printer._print)(Core.apply_type(Core.NamedTuple, (:sorted,))(Core.tuple(sorted)), TOML.Internals.Printer._print, f, io, x)
││││││││││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/TOML/src/print.jl:76 #s848(_2, _3, f, io, a, Base.getindex(TOML.Internals.Printer.String))
│││││││││││││││┌ @ /Users/aviatesk/julia/julia/usr/share/julia/stdlib/v1.7/TOML/src/print.jl:76 Core.throw(Core.UndefKeywordError(:by))
││││││││││││││││ UndefKeywordError: keyword argument by not assigned
│││││││││││││││└────────────────────────────────────────────────────────────────────────────────
Pkg.API.ProjectInfo
</details>

With this PR, everything should work:
> After
```julia
julia> TOML.print(data; sorted=true) do x
           x isa MyStruct && return Dict(:bar => x.a)
       end
[foo]
bar = 1
```
* Add adjoint for Cholesky

* Implement adjoint for BunchKaufman

* Fix ldiv! for adjoints of Hessenbergs

* Add adjoint of LDLt

* Fix return for tall problems in fallback \ method for adjoint of
Factorizations to make \ work for adjoint LQ.

* Fix qr(A)'\b

* Define adjoint for SVD

* Improve promotion in fallback by defining general convert methods
for Factorizations

* Fix ldiv! for SVD

* Restrict the general \ definition that handles over- and underdetermined
systems to LAPACK factorizations

* Remove redundant \ definitions in diagonal.jl

* Add Factorization constructors for SVD

* Disambiguate between the specialized \ for real lhs-complex rhs and
then new \ for LAPACKFactorizations.

* Adjustments based on review

* Fixes for new pivoting syntax
@tpapp
Copy link
Contributor Author

tpapp commented May 31, 2021

Friendly ping: I am wondering if someone could review this so it could get merged. It is a small improvement fixing an error message.

@KristofferC
Copy link
Member

CI seems to fail but not sure it is related. Perhaps you could rebase on master.

@vtjnash
Copy link
Member

vtjnash commented May 31, 2021

The CI failure is related: it is noting that you are requesting it to assign a value to T in the where clause, but that the value might be undefined.

tpapp added 4 commits June 3, 2021 13:56
Seeing implementation like `Base.OneTo` in error messages may be
confusing to some users (cf discussion in JuliaLang#39242,
[discourse](https://discourse.julialang.org/t/promote-shape-dimension-mismatch/57529/)).

This PR turns
```julia
julia> ones(2, 3) + ones(3, 2)
ERROR: DimensionMismatch("dimensions must match: a has dims (Base.OneTo(2), Base.OneTo(3)), b has dims (Base.OneTo(3), Base.OneTo(2)), mismatch at 1")
```
into
```julia
julia> ones(2, 3) + ones(3, 2)
ERROR: DimensionMismatch("dimensions must match: a has axes (1:2, 1:3), b has axes (1:3, 1:2), mismatch at 1")
```

Fixes JuliaLang#40118.

Acked-by: Tamas K. Papp <[email protected]>
@tpapp tpapp requested a review from a team as a code owner June 3, 2021 12:01
@tpapp
Copy link
Contributor Author

tpapp commented Jun 3, 2021

Made a mess of rebasing, will cherry-pick into another PR.

@tpapp tpapp closed this Jun 3, 2021
vtjnash added a commit that referenced this pull request Feb 4, 2024
Seeing implementation details like `Base.OneTo` in error messages may
be confusing to some users (cf discussion in #39242,
[discourse](https://discourse.julialang.org/t/promote-shape-dimension-mismatch/57529/)).

This PR turns
```julia
julia> ones(2, 3) + ones(3, 2)
ERROR: DimensionMismatch("dimensions must match: a has dims (Base.OneTo(2), Base.OneTo(3)), b has dims (Base.OneTo(3), Base.OneTo(2)), mismatch at 1")
```
into
```julia
julia> ones(2, 3) + ones(3, 2)
ERROR: DimensionMismatch("dimensions must match: a has size (2, 3), b has size (3, 2), mismatch at 1")
```

Fixes #40118. 

(This is basically #40124, but redone because I made a mess rebasing).

---------

Co-authored-by: Jameson Nash <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
error messages Better, more actionable error messages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Promote_shape dimension mismatch print.