fix build on go 1.17 and later: panic: qtls.ClientHelloInfo doesn't match

[milahu]

fix #156
fix #157
fix #122

this (patching vendor/) is the simple way
and it works = both forward and reverse migrations finish without error
currently, migrations work from version 7 to 12

# warning: revert may cause data loss

path=/home/user/.ipfs

for a in $(seq 11 -1 7); do
  echo a=$a
  ./fs-repo-$a-*/fs-repo-* -path=$path -verbose=true -revert
  sleep 2 # workaround. fixme: migration should be sync
done

for a in $(seq 7 11); do
  echo a=$a
  ./fs-repo-$a-*/fs-repo-* -path=$path -verbose=true
  sleep 2 # workaround. fixme: migration should be sync
done

a=11
reverting 11-to-12 repo migration
locking repo at "/home/user/.ipfs"
  - verifying version is '12'
  - starting raw multihash to CIDv1 block migration
  - loading repo configurations
  - backup file will be read from /home/user/.ipfs/11-to-12-cids.txt
MFS Root: QmeY5vvgBJED4GsiHe35ubCmeGwiRFkA92gFaH6EEunxzg
2023-01-11 20:31:57.92946784 +0100 CET m=+12.757777763: Generic migration worker syncing after 839 objects migrated
2023-01-11 20:31:57.931535458 +0100 CET m=+12.759845342: Generic migration worker syncing after 839 objects migrated
839 multihashes swapped to CidV1s
reverted version file to version 11
a=10
reverting migration
opening datastore at "/home/user/.ipfs"
reverting pinning to use ipld storage
converted 22 pins from datastore to ipld storage
updated version file
a=9
reverting migration
lowered version number to 9
a=8
reverting migration
Renaming key's filename:  key_gezeim2ln5xvorstmvfvcr3ynzzhqrcljq4gqqlpljifqwtoojjw26tkpbghsqlfgzivordvonbuetkbjbqq
updated version file
a=7
reverting migration
lowered version number to 7

a=7
applying 7-to-8 repo migration
locking repo at "/home/user/.ipfs"
  - verifying version is '7'
> Upgrading config to new format
updated version file
Migration 7 to 8 succeeded
a=8
applying 8-to-9 repo migration
Renaming key's filename:  12D3KooWFSeKQGxnrxDKL8hAoZPXZnrSmzjxLyAe6QWDusCBMAHa
updated version file
Migration 8 to 9 succeeded
a=9
applying 9-to-10 repo migration
locking repo at "/home/user/.ipfs"
  - verifying version is '9'
> Upgrading config to new format
updated version file
Migration 9 to 10 succeeded
a=10
applying 10-to-11 repo migration
opening datastore at "/home/user/.ipfs"
upgrading pinning to use datastore
converted 22 pins from ipld storage into datastore
updated version file
Migration 10 to 11 succeeded
a=11
applying 11-to-12 repo migration
locking repo at "/home/user/.ipfs"
  - verifying version is '11'
  - loading repo configurations
  - starting CIDv1 to raw multihash block migration
  - backup file will be written to /home/user/.ipfs/11-to-12-cids.txt
  - Adding keys in prefix /blocks to backup file
could not parse /blocks/CEKKA3LXSBZOAHTTLJKUCLOGE2IABGZA5B7Q as a Cid
could not parse /blocks/DMQD5OBY4XKNEVVELXQVQTATPG3SYQJHFF4ZZ4XDZ6TULT345YVWRMI as a Cid
could not parse /blocks/CEKNYI7757GS72XPQA3P3JC6SSNP3BVK4XLQ as a Cid
could not parse /blocks/CEKFAJPXNH2ASKNWXQ27ILO6QRLBH23OWZPQ as a Cid
458 CIDv1 keys added to backup file for /blocks
  - Adding keys in prefix /filestore to backup file
0 CIDv1 keys added to backup file for /filestore
  - backup file will be read from /home/user/.ipfs/11-to-12-cids.txt
2023-01-11 20:32:44.974778209 +0100 CET m=+0.533970302: Migration worker has moved 458 flatfs files and 458 in total
2023-01-11 20:32:44.974941058 +0100 CET m=+0.534133115: Generic migration worker syncing after 0 objects migrated
2023-01-11 20:32:44.974997261 +0100 CET m=+0.534189293: Generic migration worker syncing after 0 objects migrated
458 CidV1s swapped to multihashes
updated version file

7-to-6 fails

a=6
./fs-repo-$a-*/fs-repo-* -path=$path -verbose=true -revert
# error: expected ed25519 data size to be 96

fs-repo-migrations/fs-repo-6-to-7/gx/ipfs/QmaPbCnUMBohSGo3KnxEa2bHqyJVVeEEcwtqJAYxerieBo/go-libp2p-crypto/ed25519.go

Line 122 in 6c9f7fc

return nil, fmt.Errorf("expected ed25519 data size to be 96")

this (patching vendor/) is the simple way

i also tried the hard way, trying to update the quic-go dependency, but failed

cd fs-repo-migrations
cd fs-repo-11-to-12

go mod graph | grep github.com/lucas-clemente/[email protected]
# github.com/ipfs/[email protected] github.com/lucas-clemente/[email protected]
# github.com/libp2p/[email protected] github.com/lucas-clemente/[email protected]

# update to [email protected]
go mod graph | grep github.com/lucas-clemente/quic-go
# v0.19.3
go mod edit -replace github.com/lucas-clemente/quic-go=github.com/lucas-clemente/[email protected]
go get github.com/lucas-clemente/[email protected]
go mod vendor
# error
# github.com/libp2p/go-libp2p-quic-transport imports
# github.com/lucas-clemente/quic-go/metrics: no required module provides package github.com/lucas-clemente/quic-go/metrics

# update to [email protected]
go mod graph | grep github.com/libp2p/go-libp2p-quic-transport
# v0.10.0
go mod edit -replace github.com/libp2p/go-libp2p-quic-transport=github.com/libp2p/[email protected]
go get github.com/libp2p/[email protected]
go mod vendor
go build
# error: [email protected] is not compatible with [email protected]
# vendor/github.com/libp2p/go-libp2p-quic-transport/stream.go:12:13: undefined: quic.StreamErrorCode

# rollback
rm -rf vendor/
git reset --hard origin/master

# update to [email protected]
go mod graph | grep github.com/libp2p/go-libp2p-quic-transport
# v0.10.0
go mod edit -replace github.com/libp2p/go-libp2p-quic-transport=github.com/libp2p/[email protected]
go get github.com/libp2p/[email protected]
go mod graph | grep "[email protected]$"
# github.com/ipfs/[email protected] github.com/lucas-clemente/[email protected]
# update to [email protected]
go mod edit -replace github.com/lucas-clemente/quic-go=github.com/lucas-clemente/[email protected]
go get github.com/lucas-clemente/[email protected]
go mod graph | grep "[email protected]$"
# github.com/ipfs/[email protected] github.com/lucas-clemente/[email protected]
# so, either go-ipfs still depends on the old quic-go,
# or this is a bug in "go mod graph"

# update to [email protected]
go mod edit -replace github.com/libp2p/go-libp2p-quic-transport=github.com/libp2p/[email protected]
go get github.com/libp2p/[email protected]
go mod graph | grep quic-go
# github.com/libp2p/[email protected] github.com/lucas-clemente/[email protected]
# update to [email protected]
go mod edit -replace github.com/lucas-clemente/quic-go=github.com/lucas-clemente/[email protected]
go get github.com/lucas-clemente/[email protected]
go mod graph | grep "[email protected]$"
# github.com/ipfs/[email protected] github.com/lucas-clemente/[email protected]
# so, either go-ipfs still depends on the old quic-go,
# or this is a bug in "go mod graph"
go mod vendor
# error: golang.org/x/text/encoding: missing go.sum entry for module providing package golang.org/x/text/encoding (imported by golang.org/x/net/html/charset)

i am aware that migration is rarely needed
but at the same time, its silly to not support such a trivial operation

[Jorropo]

@milahu we don't maintain older migrations (except for migration bugs), due to the go crypto lib not exposing the required fields to implement QUIC's TLS we have to use unsafe to access private fields of the crypto api.
The crypto API internals may break from version to version so this need manual intervention from the go-quic maintainers to update each time.

TL;DR you need to use the go compiler version that was current at the time when we built the migration, while this issue in the go crypto std is not fixed.
I belive quic-go/quic-go#2727 tracks progress on this issue.