-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
deprecate (then remove) generalized linear indexing #14770
Comments
The last index is effectively interpreted as a linear index, if the dimensionality is insufficient. (More properly, for Relevant code: Line 508 in 0c20e64
LinearSlow (expands to full dimensionality): Lines 525 to 551 in 0c20e64
|
Whether the functionality is useful is not quite the same question as whether this should be the default behaviour. There could be a function |
FWIW, I tend to feel like this behavior is too odd to be the default. |
I would say that I'm looking for reconsideration, yes. I think its got a high surprising-ness to it, relative to its (minimal-but-nonzero) utility. I guess my main initial goal was understanding how much utility it does have, and to whether "fixing it" would have nontrivial performance applications. In terms of options for "fixing it", after I admit relatively little consideration, it doesn't seem like anything but requiring all dimensions would be consistent. e.g. consider
In some ways I find that even more surprising than my other example (but would have found it returning |
I agree and feel strongly that it should be an error to index an array with fewer indices than the rank of the array. It just makes it too easy to introduce hard to find errors into code when using higher rank arrays (say 4-10 dimensional). It is easy enough to use reshape to make a copy free reference if one wants to use the trailing linear indexing feature (and this leaves the intent clear in the code). And one could easily write some helper functions to make trailing linear indexing easier to use. I would also note that other languages (e.g., IDL and Fortran) make this an error so the present behavior will be surprising to many. The case of 1-D linear indexing is perhaps OK since indexing as a one dimensional array is visually distinct (compared with, e.g., indexing a 8-D array as 7-D). However, since it is so easy to use This was discussed a bit in JuliaLang/LinearAlgebra.jl#42 but I don't know how to link to specific comments and that is a long issue. |
@BobPortmann you can link to specific comments by right clicking on the timestamp (next to the author name in the header of the comment), and selecting "copy link location" or equivalent. |
+1 to reconsidering this, it just seems so much more likely that this would
be used by mistake than on purpose.
Even for linear indexing I think that we should consider if it should have
its own method instead of being overloaded onto regular indexing.
|
I kind of love the idea of giving linear indexing a special method. |
What about |
I'd be ok with that. |
+1 to dropping this behavior and having separate syntax of some kind for linear indexing. This behavior was just taken as the default choice since it's what some other environments do. I would describe it as an over-generalization of a premature optimization. |
Precisely. Let's ditch this for sure. |
-1 for dropping the 1D version. Its very natural to loop over all elements in a multidimensional array using
I know this can be achieve in other ways (e.g. enumerate) but the above is quite easy to remember. |
@JeffBezanson, i disagree with the statement about over-generalization of a premature optimization. Linear indexing is the natural indexing scheme and anything providing multi dimensional access is actually some infrastructure to make it easier for the person writing code or design an algorithm. Now to the interesting problem that started this: Having a a[i,j] access when a is actually defined 3D. I was not aware that this existed, but actually this is something that i'd like to have in a language. The initial problem for me (and i think the title of the issue is missleading) is rather in the area of a compiler warning or some code checking (like lint). |
The way I see it, linear indexing is the natural indexing scheme in the same sense that machine code is the natural programming language: it is the fundamental one. But Julia is in the business of providing abstractions. In fact, it provides abstractions such as subarrays, where linear indexing is not the natural indexing scheme. I don't think that linear indexing should be removed completely, especially not for the array types where it is the fundamental scheme. But seeing as subarray performance is going to be increasingly important, it might make sense to make it a bit more cumbersome to use linear indexing, in order to nudge people into using |
Just thinking out loudly: In addition to
This reference could be something like a zero-dimensional subarray. |
Wouldn't that have to be To just access the value, we already have that from |
Right, should have been Yes, this is about avoiding having to handle indices at all if one wants to iterate over a whole array, assigning to elements. |
That could be useful.
|
👍 to deprecating this. I've called this "partial" linear indexing before — not sure where I picked up that terminology. It's been discussed in #13015 and #5396, and it's a bullet point that I listed as a possibility in JuliaLang/LinearAlgebra.jl#255. |
That kind of iterator would be very useful for sparse matrices where accessing non-structural locations is not allowed (for example, PETSc matrices). |
i'm just wondering, how and why this was brought in, anyway? Someone found it helpful and i guess it was happening <v0.2. Maybe i'm biased because my matlab coding includes very often linear indexing although the data is organized in 2or3D and i have a mental model to deal with it. |
Another -1 for dropping the 1D version for same reason as #14770 (comment). Just keep the 1D version of linear indexing and remove the "partial" linear indexing imo. |
I too would love to have the 1d linear indexing version and getting rid of the partial linear indexing. That is quite a sane solution. |
Sorry for hijacking the thread, no matter what happens to linear indexing,
it seems that almost everyone is for getting rid of partial linear
indexing. That should also be a much smaller change, since there shouldn't
be much (if any) code that depends on it.
|
@toivoh, No, i'm not for getting rid of this partial linear indexing. |
That's why I said almost everyone :) There is of course always a tension between catching bugs and convenience, |
s/linear/partial linear/ in the last sentence.
|
Well, although I do not want to vote in a particular direction, there are use cases and I even have uses it in some situations. Imagine you have some plotting tool that is capable of displaying slices within a 3D dataset
where |
For what it's worth, the most straight-forward implementation here requires #18457 since it fixes a method sorting bug with Varargs of a bound length. |
Since #18457 is almost ready (can we merge it yet?), let's wait for that and then do that. |
An update is in order here. We've been talking about two completely different things in this thread, with vastly different ramifications:
I'm for the first change, but against the second. |
Re. indexing with trailing singletons: Though indexing with trailing singletons is sometimes convenient when writing code, that style is often confusing / obfuscating when reading code. Code being read more than written, the benefit of disallowing indexing with trailing singletons (clarity) seems worth the cost (minor convenience reduction). Deprecating indexing with anything other than |
I'm not as certain on the consensus since the thread is a little meandering… and I know I have been imprecise in my language when talking about these two options. It'd be great to hear updated opinions now that we have two very concrete (and implemented) options. You can try them out! CC @andreasnoack, who is conspicuously absent here. My hesitancy about only-1-or-N-indices isn't just that it requires a lot of changes to existing code, but that it is also more difficult to implement and enforce. Were this more consistently a net simplification (on either side), then I think I'd find it more attractive. |
@mbauman Either of your 2 options above will be a big improvement but I think having stricter rules would be better (i.e., deprecating indexing with anything other than 1 or N dimensions). It is not clear to me why this would be "more difficult to implement and enforce". Seems naively that it would be easier, not harder. In any case, in my experience this distinction only becomes an issue when working with higher-dimensional arrays (say larger that 5 or so) where being off by one index is less visually distinctive and getting yelled at by the compiler really helps in the long run. I suppose the linear-algebra folks usually use lower dimensional arrays and thus have no issue. |
I can't speak to the difficulty of implementation, but if we're prioritizing user experience, unintentionally providing the wrong number of indices is currently an unfortunate trap. We get complaints about this in JuMP: jump-dev/JuMP.jl#937 |
I'm also in in favor of 1 or N. One of the other people in my lab spent nearly a week tracking down a bug caused by this behavior. |
Moving milestone now that #20079 is merged. |
What remains to be done here? Now is probably a good time to take this out entirely and try to simplify all the array indexing infrastructure. |
I need to spend some time to get #21750 through. Then we can talk about the next steps here. I'd be in favor of one final round of bounds check tightening here, such that we only allow omitted trailing dimensions when the size of those omitted dimensions are all |
With #21750 merged, I believe there are only a few small indexing cases that still need to be deprecated here. |
I'm sure this has been discussed elsewhere, but maybe not recently or in a focussed/isolated fashion. If this shouldn't be an issue report, I'll move it to julia-users.
Essentially, I would say this behavior is surprising:
This was a cause of a hard-to-find bug for me recently, where I accidentally forgot an index.
I suppose this is in some way a generalization of the fact that
x[12]
works, but I'm not really sure why or where that behavior is useful for 2D-or-higher indices. Is there some logic for the current behavior?EDIT: most similar discussion: #5396
The text was updated successfully, but these errors were encountered: