Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: run migrations on restored KV database before swapping it #21868

Merged
merged 4 commits into from
Jul 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,13 @@ Because of the version bump to `go`, the macOS build for this release requires a

### Bug Fixes

1. [21748](https://github.com/influxdata/influxdb/pull/21748): rename arm rpms with yum-compatible names.
1. [21748](https://github.com/influxdata/influxdb/pull/21748): Rename arm rpms with yum-compatible names.
1. [21851](https://github.com/influxdata/influxdb/pull/21851): Upgrade to latest version of `influxdata/cron` so that tasks can be created with interval of `every: 1w`.
1. [21859](https://github.com/influxdata/influxdb/pull/21859): Avoid rewriting `fields.idx` unnecessarily.
1. [21860](https://github.com/influxdata/influxdb/pull/21860): Do not close connection twice in DigestWithOptions.
1. [21866](https://github.com/influxdata/influxdb/pull/21866): Remove incorrect optimization for group-by.
1. [21867](https://github.com/influxdata/influxdb/pull/21867): Return an error instead of panicking when InfluxQL statement rewrites fail.
1. [21868](https://github.com/influxdata/influxdb/pull/21868): Migrate restored KV snapshots to latest schema before using them.

## v2.0.7 [2021-06-04]
----------------------
Expand Down
34 changes: 34 additions & 0 deletions bolt/kv.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ import (

"github.com/influxdata/influxdb/v2/kit/tracing"
"github.com/influxdata/influxdb/v2/kv"
"github.com/influxdata/influxdb/v2/kv/migration"
"github.com/influxdata/influxdb/v2/kv/migration/all"
"github.com/influxdata/influxdb/v2/pkg/fs"
bolt "go.etcd.io/bbolt"
"go.uber.org/zap"
Expand Down Expand Up @@ -221,6 +223,11 @@ func (s *KVStore) Restore(ctx context.Context, r io.Reader) error {
return err
}

// Run the migrations on the restored database prior to swapping it in.
if err := s.migrateRestored(ctx); err != nil {
return err
}

// Swap and reopen under lock.
s.mu.Lock()
defer s.mu.Unlock()
Expand All @@ -243,6 +250,33 @@ func (s *KVStore) Restore(ctx context.Context, r io.Reader) error {
return nil
}

// migrateRestored opens the database at the temporary path and applies the
// migrations to it. The database at the temporary path is closed after the
// migrations are complete. This should be used as part of the restore
// operation, prior to swapping the restored database with the active database.
func (s *KVStore) migrateRestored(ctx context.Context) error {
restoredClient := NewClient(s.log.With(zap.String("service", "restored bolt")))
restoredClient.Path = s.tempPath()
if err := restoredClient.Open(ctx); err != nil {
return err
}
defer restoredClient.Close()

restoredKV := NewKVStore(s.log.With(zap.String("service", "restored kvstore-bolt")), s.tempPath())
restoredKV.WithDB(restoredClient.DB())

migrator, err := migration.NewMigrator(
s.log.With(zap.String("service", "bolt restore migrations")),
restoredKV,
all.Migrations[:]...,
)
if err != nil {
return err
}

return migrator.Up(ctx)
}

// Tx is a light wrapper around a boltdb transaction. It implements kv.Tx.
type Tx struct {
tx *bolt.Tx
Expand Down