Skip to content

Commit 4f5e58c

Browse files
meiji163grodowski
andauthored
Panic if InitiateHeartbeat exhausts retries to avoid looping infinitely (#1594)
* Panic if InitiateHeartbeat exhausts retries to avoid looping infinitely. Based on experience, if the writer database fails inbeetween the copy & cutover stages (e.g. during cutover pause), the heartbeat writes will fail and stop, then leading to throttled state and an infinite loop of throttler.shouldThrottle(). Since this state is irrecoverable, make the heartbeat writer panic if retries are exhausted, so that the migration can fail and be restarted later. * Add sysbench localtest (#1590) * add sysbench localtest * fix table name * Apply suggestion from @Copilot Co-authored-by: Copilot <[email protected]> * ensure cleanup --------- Co-authored-by: Copilot <[email protected]> * add toxiproxy option for localtests (#1591) --------- Co-authored-by: Jan Grodowski <[email protected]>
1 parent 6b676f1 commit 4f5e58c

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

go/logic/applier.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -575,6 +575,7 @@ func (this *Applier) InitiateHeartbeat() {
575575
continue
576576
}
577577
if err := injectHeartbeat(); err != nil {
578+
this.migrationContext.PanicAbort <- fmt.Errorf("injectHeartbeat writing failed %d times, last error: %w", numSuccessiveFailures, err)
578579
return
579580
}
580581
}

0 commit comments

Comments
 (0)