May 7, 2024 @ 09:40:24	drbd filesystem g2: Committing remote state change 36291832 (primary_nodes=2)
May 7, 2024 @ 09:40:24	drbd filesystem g2: Preparing remote state change 36291832
May 7, 2024 @ 09:40:24	drbd images/0 drbd2: FIXME swapper/27[0] op set, bitmap locked for 'send_bitmap (WFBitMapS)' by drbd_w_images[1799]
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: Began resync as SyncSource (will sync 53252 KB [13313 bits set]).
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: drbd_sync_handshake:
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: helper command: /sbin/drbdadm before-resync-source
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: helper command: /sbin/drbdadm before-resync-source exit code 0
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: pdsk( DUnknown -> Outdated ) repl( Off -> WFBitMapS ) [connected]
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: pdsk( Outdated -> Inconsistent ) repl( WFBitMapS -> SyncSource ) [receive-bitmap]
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: peer 9C47B5781571369A:0000000000000000:2D2FF06C5ED4C144:295233C594AED602 bits:0 flags:0
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 3360(1), total 3360; compression: 100.0%
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: repl( SyncSource -> PausedSyncS ) resync-susp( no -> peer ) [peer-state]
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: Resync suspended
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: self 3CBF0F4804B65A0C:9C47B5781571369A:8AB3ED878817F97E:2D2FF06C5ED4C144 bits:13300 flags:100
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 3360(1), total 3360; compression: 100.0%
May 7, 2024 @ 09:40:24	drbd images/0 drbd2 g3: uuid_compare()=source-use-bitmap by rule=bitmap-self
May 7, 2024 @ 09:40:24	drbd images: Committing cluster-wide state change 3374929522 (37ms)
May 7, 2024 @ 09:40:24	drbd images g2: Committing remote state change 3518791310 (primary_nodes=4)
May 7, 2024 @ 09:40:24	drbd images g2: Preparing remote state change 3518791310
May 7, 2024 @ 09:40:24	drbd images g3: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected]
May 7, 2024 @ 09:40:24	drbd images g3: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG
May 7, 2024 @ 09:40:24	drbd images g3: Handshake to peer 3 successful: Agreed network protocol version 122
May 7, 2024 @ 09:40:24	drbd images: Preparing cluster-wide state change 3374929522 (1->3 499/146)
May 7, 2024 @ 09:40:24	drbd images: State change 3374929522: primary_nodes=4, weak_nodes=FFFFFFFFFFFFFFF1
May 7, 2024 @ 09:40:25	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:26	drbd images/0 drbd2: bitmap WRITE of 369 pages took 18 ms
May 7, 2024 @ 09:40:26	drbd images/0 drbd2 g3: pdsk( Inconsistent -> UpToDate ) repl( PausedSyncS -> Established ) [resync-finished]
May 7, 2024 @ 09:40:26	drbd images/0 drbd2 g3: Resync done (total 2 sec; paused 0 sec; 26624 K/sec)
May 7, 2024 @ 09:40:26	drbd images/0 drbd2 g3: resync-susp( peer -> no ) [peer-state]
May 7, 2024 @ 09:40:26	drbd images/0 drbd2 g3: updated UUIDs 3CBF0F4804B65A0C:0000000000000000:9C47B5781571369A:8AB3ED878817F97E
May 7, 2024 @ 09:40:26	drbd images rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:28	drbd filesystem/0 drbd1 g3: drbd_sync_handshake:
May 7, 2024 @ 09:40:28	drbd filesystem/0 drbd1 g3: peer A71DD768F1F43406:490D556F1639F082:D2329499827A79D4:D92FB18EB968A51E bits:61404 flags:1824
May 7, 2024 @ 09:40:28	drbd filesystem/0 drbd1 g3: self A71DD768F1F43407:490D556F1639F083:FBD1BE70A901157C:D2329499827A79D4 bits:373616 flags:120
May 7, 2024 @ 09:40:28	drbd filesystem/0 drbd1 g3: strategy = source-use-bitmap due to disk states. (UpToDate/Inconsistent)
May 7, 2024 @ 09:40:28	drbd filesystem/0 drbd1 g3: uuid_compare()=no-sync by rule=lost-quorum
May 7, 2024 @ 09:40:28	drbd filesystem g3: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG
May 7, 2024 @ 09:40:28	drbd filesystem g3: Handshake to peer 3 successful: Agreed network protocol version 122
May 7, 2024 @ 09:40:28	drbd filesystem: Preparing cluster-wide state change 1043184996 (1->3 499/145)
May 7, 2024 @ 09:40:28	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:29	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem/0 drbd1: FIXME swapper/27[0] op clear, bitmap locked for 'send_bitmap (WFBitMapS)' by drbd_w_filesyst[1797]
May 7, 2024 @ 09:40:32	drbd filesystem/0 drbd1 g3: pdsk( DUnknown -> Inconsistent ) repl( Off -> WFBitMapS ) resync-susp( no -> peer ) [connected]
May 7, 2024 @ 09:40:32	drbd filesystem: Committing cluster-wide state change 1043184996 (4301ms)
May 7, 2024 @ 09:40:32	drbd filesystem g3: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected]
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:32	drbd filesystem: State change 1043184996: primary_nodes=2, weak_nodes=FFFFFFFFFFFFFFF1
May 7, 2024 @ 09:40:33	drbd filesystem/0 drbd1 g3: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 32257(8), total 32257; compression: 100.0%
May 7, 2024 @ 09:40:33	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:33	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:33	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:37	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!
May 7, 2024 @ 09:40:39	drbd filesystem rdma:g2: Not sending flow_control mgs, no receive window!


-- timeout start approximately at 09:41:12
May 7, 2024 @ 09:41:04	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 6
May 7, 2024 @ 09:41:08	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 5
May 7, 2024 @ 09:41:12	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 4
May 7, 2024 @ 09:41:16	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 3
May 7, 2024 @ 09:41:20	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 2
May 7, 2024 @ 09:41:25	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 1
May 7, 2024 @ 09:41:28	drbd filesystem/0 drbd1 g2: We did not send a P_BARRIER for 28027ms > ko-count (7) * timeout (40 * 0.1s); drbd kernel thread blocked?
May 7, 2024 @ 09:41:33	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 6
May 7, 2024 @ 09:41:37	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 5
May 7, 2024 @ 09:41:41	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 4
May 7, 2024 @ 09:41:45	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 3
May 7, 2024 @ 09:41:49	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 2
May 7, 2024 @ 09:41:53	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 1
May 7, 2024 @ 09:41:58	drbd filesystem/0 drbd1 g2: We did not send a P_BARRIER for 28032ms > ko-count (7) * timeout (40 * 0.1s); drbd kernel thread blocked?
May 7, 2024 @ 09:42:01	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 6
May 7, 2024 @ 09:42:05	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 5
May 7, 2024 @ 09:42:09	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 4
May 7, 2024 @ 09:42:13	drbd filesystem/0 drbd1 g3: resync-susp( peer -> no ) [peer-state]
May 7, 2024 @ 09:42:13	drbd filesystem g3: [drbd_s_filesyst/4187945] sending time expired, ko = 3
May 7, 2024 @ 09:42:14	drbd filesystem/0 drbd1 g3: Began resync as SyncSource (will sync 1276048 KB [319012 bits set]).
May 7, 2024 @ 09:42:14	drbd filesystem/0 drbd1 g3: helper command: /sbin/drbdadm before-resync-source
May 7, 2024 @ 09:42:14	drbd filesystem/0 drbd1 g3: helper command: /sbin/drbdadm before-resync-source exit code 0
May 7, 2024 @ 09:42:14	drbd filesystem/0 drbd1 g3: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 32366(8), total 32366; compression: 100.0%
May 7, 2024 @ 09:42:14	drbd filesystem/0 drbd1 g3: repl( WFBitMapS -> SyncSource ) [receive-bitmap]
May 7, 2024 @ 09:43:38	drbd filesystem/0 drbd1 g3: Remote failed to answer a P_BARRIER (sent at 9217587200 jif; now=9217621504 jif) within 34304ms > ko-count (7) * timeout (40 * 0.1s)
May 7, 2024 @ 09:43:38	drbd filesystem/0 drbd1 g3: repl( SyncSource -> Off ) [timeout]
May 7, 2024 @ 09:43:38	drbd filesystem g3: conn( Connected -> Timeout ) peer( Secondary -> Unknown ) [timeout]
May 7, 2024 @ 09:43:38	drbd filesystem g3: Connection closed
May 7, 2024 @ 09:43:38	drbd filesystem g3: conn( Timeout -> Unconnected ) [disconnected]
May 7, 2024 @ 09:43:38	drbd filesystem g3: conn( Unconnected -> Connecting ) [connecting]
May 7, 2024 @ 09:43:38	drbd filesystem g3: Feature flags enabled on protocol level: 0x7f TRIM THIN_RESYNC WRITE_SAME WRITE_ZEROES RESYNC_DAGTAG
May 7, 2024 @ 09:43:38	drbd filesystem g3: Handshake to peer 3 successful: Agreed network protocol version 122
May 7, 2024 @ 09:43:38	drbd filesystem g3: helper command: /sbin/drbdadm disconnected
May 7, 2024 @ 09:43:38	drbd filesystem g3: helper command: /sbin/drbdadm disconnected exit code 0
May 7, 2024 @ 09:43:38	drbd filesystem g3: meta connection shut down by peer.
May 7, 2024 @ 09:43:38	drbd filesystem g3: Restarting receiver thread
May 7, 2024 @ 09:43:38	drbd filesystem g3: sock_recvmsg returned -4
May 7, 2024 @ 09:43:38	drbd filesystem g3: Starting sender thread (from drbd_r_filesyst [246159])
May 7, 2024 @ 09:43:38	drbd filesystem g3: Terminating sender thread
May 7, 2024 @ 09:43:38	drbd filesystem: Preparing cluster-wide state change 952409516 (1->3 499/145)
May 7, 2024 @ 09:43:39	drbd filesystem/0 drbd1 g3: drbd_sync_handshake:
May 7, 2024 @ 09:43:39	drbd filesystem/0 drbd1 g3: peer 490D556F1639F082:490D556F1639F082:D2329499827A79D4:D92FB18EB968A51E bits:28325 flags:1124
May 7, 2024 @ 09:43:39	drbd filesystem/0 drbd1 g3: repl( Off -> WFBitMapS ) [connected]
May 7, 2024 @ 09:43:39	drbd filesystem/0 drbd1 g3: self A71DD768F1F43407:490D556F1639F083:FBD1BE70A901157C:D2329499827A79D4 bits:29670 flags:120
May 7, 2024 @ 09:43:39	drbd filesystem/0 drbd1 g3: send bitmap stats [Bytes(packets)]: plain 0(0), RLE 12865(4), total 12865; compression: 100.0%
May 7, 2024 @ 09:43:39	drbd filesystem/0 drbd1 g3: uuid_compare()=source-use-bitmap by rule=bitmap-self
May 7, 2024 @ 09:43:39	drbd filesystem: Committing cluster-wide state change 952409516 (393ms)
May 7, 2024 @ 09:43:39	drbd filesystem g3: conn( Connecting -> Connected ) peer( Unknown -> Secondary ) [connected]
May 7, 2024 @ 09:43:39	drbd filesystem: State change 952409516: primary_nodes=2, weak_nodes=FFFFFFFFFFFFFFF1
May 7, 2024 @ 09:43:40	drbd filesystem/0 drbd1 g3: Began resync as SyncSource (will sync 125820 KB [31455 bits set]).
May 7, 2024 @ 09:43:40	drbd filesystem/0 drbd1 g3: helper command: /sbin/drbdadm before-resync-source
May 7, 2024 @ 09:43:40	drbd filesystem/0 drbd1 g3: helper command: /sbin/drbdadm before-resync-source exit code 0
May 7, 2024 @ 09:43:40	drbd filesystem/0 drbd1 g3: receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 12925(4), total 12925; compression: 100.0%
May 7, 2024 @ 09:43:40	drbd filesystem/0 drbd1 g3: repl( WFBitMapS -> SyncSource ) [receive-bitmap]
May 7, 2024 @ 09:43:57	drbd filesystem/0 drbd1 g3: updated UUIDs A71DD768F1F43407:0000000000000000:490D556F1639F082:FBD1BE70A901157C
May 7, 2024 @ 09:44:00	drbd filesystem/0 drbd1 g3: pdsk( Inconsistent -> UpToDate ) repl( SyncSource -> Established ) [resync-finished]
May 7, 2024 @ 09:44:00	drbd filesystem/0 drbd1 g3: Resync done (total 20 sec; paused 0 sec; 6288 K/sec)