Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rescale tensors and choose correct contraction order optimizer in tests #16

Merged
merged 6 commits into from
Nov 4, 2022

Conversation

GiggleLiu
Copy link
Member

In this PR, I

  • Fix a bug about tensor shape,
  • Rescale tensor networks to avoid exploding parameters, I do not have any good idea to tune the parameters automatically so far.
  • Tune contraction order optimizer in the test,

Test Report

For "relational" dataset, none of the optimizers in OMEinsum find any proper contraction order. (there are more than 60k tensors, the size is too big I think).

The rest instances all pass except linkage_15 fails and ObjectDetection_35, I still do not have any clue about why these two instances are so special.

@GiggleLiu
Copy link
Member Author

Update

Now I use the following solution to the rescaling problem:

  1. create a new array type RescaledArray
struct RescaledArray{T, N, AT<:AbstractArray{T, N}} <: AbstractArray{T, N}
    logf::T
    value::AT
end

It represents array exp(logf) * value

  1. implement einsum and some other function interfaces,
for CT in [:DynamicEinCode, :StaticEinCode]
    @eval function OMEinsum.einsum(code::$CT, @nospecialize(xs::NTuple{N,RescaledArray}), size_dict::Dict) where N
        res = einsum(code, getfield.(xs, :value), size_dict)
        return rescale_array(RescaledArray(sum(x->x.logf, xs), res))
    end
end
  1. Then the overflow problem should disappear if we use this new array type for tensor network contraction.

src/RescaledArray.jl Outdated Show resolved Hide resolved
src/RescaledArray.jl Outdated Show resolved Hide resolved
src/Core.jl Outdated Show resolved Hide resolved
src/inference.jl Outdated Show resolved Hide resolved
src/RescaledArray.jl Outdated Show resolved Hide resolved
src/RescaledArray.jl Outdated Show resolved Hide resolved
src/Core.jl Show resolved Hide resolved
@GiggleLiu GiggleLiu changed the title [WIP] Rescale tensors and choose correct contraction order optimizer in tests Rescale tensors and choose correct contraction order optimizer in tests Nov 3, 2022
@mroavi mroavi merged commit ce794bd into main Nov 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants