Code

Parallel FREEZE TABLE#164

Code

Merged

aalexfvk merged 24 commits intoyandex:mainyandex/ch-backup:mainfrom kirillgarbar:parallel-freezekirillgarbar/ch-backup:parallel-freezeCopy head branch name to clipboardJul 10, 2024

+324-96

Commits

Commits on Jun 19, 2024

• Freeze and backup thread pools

  kirillgarbar

  committed
  4c3acbf

Commits on Jun 20, 2024

• Remove freezed data for one table

  kirillgarbar

  committed
  faecbed

• Freeze workers option in config

  kirillgarbar

  committed
  8807c86

Commits on Jun 22, 2024

• More flexibility of data generation for tests

  kirillgarbar

  committed
  1e86b09

• Parallel freeze test

  kirillgarbar

  committed
  6b7182b

• Move executors to subclass

  kirillgarbar

  committed
  c45b386

• Partition by prefix to fix dedup tests

  kirillgarbar

  committed
  ec20451

Commits on Jun 24, 2024

• Fix codespell

  kirillgarbar

  committed
  8b795ae

• Fix for old python

  kirillgarbar

  committed
  4f8bfec

Commits on Jun 25, 2024

• Remove backup executor and backup in the main thread

  kirillgarbar

  committed
  7554870

• Fix black

  kirillgarbar

  committed
  06cb097

• Fix race condition with shadow/increment.txt

  kirillgarbar

  committed
  95bc77b

Commits on Jul 1, 2024

• Parallel backup

  kirillgarbar

  committed
  0214f1e

• Fix unit test - can't pickle lock

  kirillgarbar

  committed
  6933163

Commits on Jul 8, 2024

• Revert "Fix unit test - can't pickle lock"

  Show description for d0a25c9
  kirillgarbar

  committed
  d0a25c9

• Revert "Parallel backup"

  Show description for 827a66d
  kirillgarbar

  committed
  827a66d

• Backup tables after all tables are frozen

  kirillgarbar

  committed
  93da739

• Move thread pool related logic to separate class

  kirillgarbar

  committed
  b7a4358

• Move all freeze related logic to TableFreezer

  kirillgarbar

  committed
  6d1076a

• Small fixes

  kirillgarbar

  committed
  c70ad1f

• Fix exception suppression in TableFreezer's __exit__

  kirillgarbar

  committed
  190539c

Commits on Jul 9, 2024

• Fix mypy

  kirillgarbar

  committed
  59fdf82

• Fix black

  kirillgarbar

  committed
  2f12cb7

• Move freeze logic to the method

  kirillgarbar

  committed
  87f9d30