-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add modify! function for lookup/update/insert/delete in one go #33758
base: master
Are you sure you want to change the base?
Changes from 1 commit
a9d178e
43fa640
44d0078
73e1141
b1dac99
28900ad
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -465,6 +465,63 @@ function hash(a::AbstractDict, h::UInt) | |||||
hash(hv, h) | ||||||
end | ||||||
|
||||||
""" | ||||||
modify!(f, d::AbstractDict{K, V}, key) | ||||||
|
||||||
Lookup and then update, insert or delete in one go without re-computing the hash. | ||||||
|
||||||
`f` is a callable object that must accept `Union{Some{V}, Nothing}` and return | ||||||
tkf marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
`Union{T, Some{T}, Nothing}` where `T` is a type [`convert`](@ref)-able to the value type | ||||||
tkf marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
`V`. The value `Some(d[key])` is passed to `f` if `haskey(d, key)`; otherwise `nothing` | ||||||
is passed. If `f` returns `nothing`, corresponding entry in the dictionary `d` is removed. | ||||||
If `f` returns non-`nothing` value `x`, `something(x)` is inserted to `d`. | ||||||
tkf marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
`modify!` returns whatever `f` returns as-is. | ||||||
|
||||||
# Examples | ||||||
```jldoctest | ||||||
julia> dict = Dict("a" => 1); | ||||||
|
||||||
julia> modify!(dict, "a") do val | ||||||
Some(val === nothing ? 1 : something(val) + 1) | ||||||
end | ||||||
Some(2) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wonder if we should make this result There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is now implemented in 28900ad. |
||||||
|
||||||
julia> dict | ||||||
Dict{String,Int64} with 1 entry: | ||||||
"a" => 2 | ||||||
|
||||||
julia> dict = Dict(); | ||||||
|
||||||
julia> modify!(dict, "a") do val | ||||||
Some(val === nothing ? 1 : something(val) + 1) | ||||||
tkf marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
end | ||||||
Some(1) | ||||||
|
||||||
julia> dict | ||||||
Dict{Any,Any} with 1 entry: | ||||||
"a" => 1 | ||||||
|
||||||
julia> modify!(_ -> nothing, dict, "a") | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
I think that |
||||||
|
||||||
julia> dict | ||||||
Dict{Any,Any} with 0 entries | ||||||
``` | ||||||
""" | ||||||
function modify!(f, dict::AbstractDict, key) | ||||||
if haskey(dict, key) | ||||||
val = f(Some(dict[key])) | ||||||
tkf marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
else | ||||||
val = f(nothing) | ||||||
end | ||||||
if val === nothing | ||||||
delete!(dict, key) | ||||||
else | ||||||
dict[key] = something(val) | ||||||
end | ||||||
return val | ||||||
end | ||||||
|
||||||
function getindex(t::AbstractDict, key) | ||||||
v = get(t, key, secret_table_token) | ||||||
if v === secret_table_token | ||||||
|
Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -391,6 +391,45 @@ function setindex!(h::Dict{K,V}, v0, key::K) where V where K | |||||||||||||||||||||||||||||||||||||||
return h | ||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
function modify!(f, h::Dict{K}, key0) where K | ||||||||||||||||||||||||||||||||||||||||
tkf marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||||||||||||||||||||||
key = convert(K, key0) | ||||||||||||||||||||||||||||||||||||||||
if !isequal(key, key0) | ||||||||||||||||||||||||||||||||||||||||
throw(ArgumentError("$(limitrepr(key0)) is not a valid key for type $K")) | ||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
# Ideally, to improve performance for the case that requires | ||||||||||||||||||||||||||||||||||||||||
# resizing, we should use something like `ht_keyindex` while | ||||||||||||||||||||||||||||||||||||||||
# keeping computed hash value and then do something like | ||||||||||||||||||||||||||||||||||||||||
# `ht_keyindex2!` if `f` returns non-`nothing`. | ||||||||||||||||||||||||||||||||||||||||
idx = ht_keyindex2!(h, key) | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
age0 = h.age | ||||||||||||||||||||||||||||||||||||||||
if idx > 0 | ||||||||||||||||||||||||||||||||||||||||
@inbounds vold = h.vals[idx] | ||||||||||||||||||||||||||||||||||||||||
vnew = f(Some(vold)) | ||||||||||||||||||||||||||||||||||||||||
tkf marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||||||||||||||||||||||
else | ||||||||||||||||||||||||||||||||||||||||
vnew = f(nothing) | ||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||
if h.age != age0 | ||||||||||||||||||||||||||||||||||||||||
idx = ht_keyindex2!(h, key) | ||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
if vnew === nothing | ||||||||||||||||||||||||||||||||||||||||
if idx > 0 | ||||||||||||||||||||||||||||||||||||||||
_delete!(h, idx) | ||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||
else | ||||||||||||||||||||||||||||||||||||||||
if idx > 0 | ||||||||||||||||||||||||||||||||||||||||
h.age += 1 | ||||||||||||||||||||||||||||||||||||||||
@inbounds h.keys[idx] = key | ||||||||||||||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I copied this line There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What’s going on here is Any time the GC is called (to allocate) it may call finalizers which might mutate the dictionary. To protect itself it checks This works in single threaded concurrency - but I have no idea if the implementation is valid under multithreading? (@vtjnash did I get those details right?) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks! So, is it effectively There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry I didn't actually address your original question (I was thinking of the So the In the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. FYI, existing Lines 446 to 464 in 6eebbbe
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm interesting. Still not sure what it protects against... it would be ideal to hear from Jeff or Jameson, but leaving it in doesn’t hurt (except maybe performance slightly). |
||||||||||||||||||||||||||||||||||||||||
@inbounds h.vals[idx] = something(vnew) | ||||||||||||||||||||||||||||||||||||||||
else | ||||||||||||||||||||||||||||||||||||||||
@inbounds _setindex!(h, something(vnew), key, -idx) | ||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||
return vnew | ||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
""" | ||||||||||||||||||||||||||||||||||||||||
get!(collection, key, default) | ||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -514,6 +514,7 @@ export | |
mapreduce, | ||
merge!, | ||
merge, | ||
modify!, | ||
pairs, | ||
reduce, | ||
setdiff!, | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
modify!
sounds a bit like a weird name to me.update!
sounds better.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI,
update!
andset!
were suggested to be simpler shorthands of what I callmodify!
in this PR. I don't mind renaming to something else, though.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Coming from SQL nomenclature, I think
modify!
is superior toupdate!
since this operation can also mean removing an entry... not just updating an entry's value. Perhapsset!
is the best choice.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Re-visiting this naming and reading @clarkevans's comment, I actually think
modify!
is still the best name so far. I think it's better at conveying the point that this API could be used for updating, inserting, or removing an entry. In particular,update!
(and with a lesser extent,set!
) sounds a bit weak at reminding the reader of the code about this. So, my preference ismodify!
>set!
>>update!
.@nalimilan What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both "modify" and "update" refer to the dict rather than the value AFAICT, so they both cover the case where you remove the value.
set!
could be interesting due to the parallel withget!
. Not sure whether it's a good parallel or not...There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I prefer
set!
overmodify!
aesthetically, andmodify!
toupdate!
. What is the resistance of calling itset!
? It's a very interesting function. Given that it'll be most likely used in ado val .... end
block, a few more characters don't matter that much, perhaps, as andyferris suggests,set_or_delete!
. Anyway, in my mind,set!
given a function returningnothing
seems perfectly compatible with removing a key.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@clarkevans I think @nalimilan mentioned a good point that
set!
sounds like a counter part ofget
. However, I don't think it's a good parallel. If you considerset!
as a counter part ofget(::Container, ::Key) ::Union{Nothing,Some{Value}}
, the signature ofset!
should beset!(::Container, ::Union{Nothing,Some{Value}}, ::Key)
. This is howgetindex
andsetindex!
work.It is conceivable to have this
set!
API since it's very useful when you want to "rollback" a container:So, I prefer using another verb for the API for this PR.
@nalimilan But doesn't
get
refer to a "slot" rather than the dict? So, isn't it strange that the mutation API refers to the whole dict rather than a slot? Also, other mutation verbs likesetindex!
andsetproperty!
seem to refer to a slot.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well currently we have
get(f::Function, collection, key)
, whose signature is very the same asmodify!(f, d, key)
in this PR. If we addedget(collection, key)::Union{Nothing,Some}
it could make sense to addset!(collection, value::Union{Nothing,Some}}, key)
too.I find it hard to tell TBH. It's hard to argue about these things, and if we wanted a fully consistent naming scheme maybe
get
should be calledgetkey
...There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From a readability standpoint, if naively read
set!
I'd probably expect the key to exist after the operation. Which is also just likeget!
. I'd feel surprised if an operation namedset!
deleted something, to be honest!I suppose a simpler
modify!(d, key, v::Union{Nothing, Some{Value}})
could be nice. I note in other places inBase
we do seem to be struggling whether dispatching on::Function
is desirable and these arguments seemed to be getting widened to::Any
. Partly because some callable things areFunction
orCallable
(like python objects). (I was also sensing that there seemed to be some more general arguments that we should be able to know the semantics of a method by the function name and number of arguments only.)I agree - I have always read these verbs as refering to the "slot" on the container.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about
crud!
it does create, update, delete... and retrieve. There is alsomerge!
akaupsert!
from SQL. I thinkcrud!
is a new kind of word that is fun to say and perhaps reflects this hybrid semantics?