Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VStreamer: recompute table plan if a new table is encountered for the same id #9978

Merged
merged 3 commits into from
Mar 26, 2022
Merged

VStreamer: recompute table plan if a new table is encountered for the same id #9978

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
61036d3
Recompute the plan for a table if a new table is encountered for the …
rohit-nayak-ps Mar 24, 2022
bdbef62
Trigger rebuild
rohit-nayak-ps Mar 24, 2022
432e912
Trigger rebuild
rohit-nayak-ps Mar 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 13 additions & 4 deletions go/vt/vttablet/tabletserver/vstreamer/vstreamer.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -601,14 +601,23 @@ func (vs *vstreamer) parseEvent(ev mysql.BinlogEvent) ([]*binlogdatapb.VEvent, e
// will generate a new plan and FIELD event.
id := ev.TableID(vs.format)

if _, ok := vs.plans[id]; ok {
return nil, nil
}

tm, err := ev.TableMap(vs.format)
if err != nil {
return nil, err
}

if plan, ok := vs.plans[id]; ok {
// When the underlying mysql server restarts the table map can change.
// Usually the vstreamer will also error out when this happens, and vstreamer re-initializes its table map.
// But if the vstreamer is not aware of the restart, we could get an id that matches one in the cache, but
// is for a different table. We then invalidate and recompute the plan for this id.
if plan == nil || plan.Table.Name == tm.Name {
return nil, nil
}
vs.plans[id] = nil
log.Infof("table map changed: id %d for %s has changed to %s", id, plan.Table.Name, tm.Name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also note that we will reload and cache a new plan for the table to avoid concern and indicate corrective action being taken ~:

log.Infof("Table plan map changed in VStream: the id %d was associated with %s and now represents %s. We will reload and cache a new plan for the table.", id, plan.Table.Name, tm.Name)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion, I will update in my next PR though, rather than start another CI cycle for this ...

}

if tm.Database == "_vt" && tm.Name == "resharding_journal" {
// A journal is a special case that generates a JOURNAL event.
return nil, vs.buildJournalPlan(id, tm)
Expand Down