ctlv3/command: auto compact and defrag after "check perf"

[gyuho]

$ ETCDCTL_API=3 ./bin/etcdctl endpoint status -w table                                                                                 
+----------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+                                         
|    ENDPOINT    |        ID        |  VERSION  | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |                                         
+----------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+                                         
| 127.0.0.1:2379 | 8e9e05c52164694d | 3.3.0+git |   20 kB |      true |         2 |          4 |                  4 |        |                                         
+----------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+                                         

$ ETCDCTL_API=3 ./bin/etcdctl check perf --auto-compact --auto-defrag                                                                  
 60 / 60 Boooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00%1m0s
Compacting with revision 8994            
Compacted with revision 8994             
Defragmenting "127.0.0.1:2379"           
Defragmented "127.0.0.1:2379"            
PASS: Throughput is 150 writes/s         
PASS: Slowest request took 0.146769s     
PASS: Stddev is 0.002704s                
PASS                                     

$ ETCDCTL_API=3 ./bin/etcdctl endpoint status -w table                                                                                 
+----------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+                                         
|    ENDPOINT    |        ID        |  VERSION  | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |                                         
+----------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+                                         
| 127.0.0.1:2379 | 8e9e05c52164694d | 3.3.0+git |   20 kB |      true |         2 |       8998 |               8998 |        |                                         
+----------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+      

/cc @raoofm @spzala

For #9326.

[gyuho]

@raoofm I added --auto-compact flag, as an option. It will discards all entries before the revision from last delete operation. It's optional, in case check perf commands are run mixed with other data (it will discards previous keys with previous revisions).

[spzala]

@gyuho this is very useful and it looks great. Couple of comments - 1. can we have both of these as a func on its own in util.go or at least in check.go if you think it's a good idea for re-usability? I believe I should use these options with check datascale as a follow up PR on my PR under review too and can simply call these functions. 2. I have noticed in few place in doc, that compact and defrag typically follow up with etcdctl alarm disarm, we may not need it here but just a thought if it's good to add disalarm as an option too. Thanks!

[codecov-io]

Codecov Report

Merging #9330 into master will decrease coverage by 0.36%.
The diff coverage is 9.52%.

@@            Coverage Diff             @@
##           master    #9330      +/-   ##
==========================================
- Coverage   75.81%   75.44%   -0.37%     
==========================================
  Files         365      365              
  Lines       30695    30715      +20     
==========================================
- Hits        23270    23173      -97     
- Misses       5814     5921     +107     
- Partials     1611     1621      +10

Impacted Files	Coverage Δ
etcdctl/ctlv3/command/check.go	13.44% <9.52%> (-0.7%)	⬇️
clientv3/ordering/kv.go	16.17% <0%> (-69.12%)	⬇️
pkg/transport/timeout_conn.go	80% <0%> (-20%)	⬇️
pkg/adt/interval_tree.go	85.28% <0%> (-8.71%)	⬇️
etcdserver/api/v3rpc/lease.go	78.37% <0%> (-8.11%)	⬇️
pkg/fileutil/purge.go	73.68% <0%> (-7.9%)	⬇️
clientv3/namespace/watch.go	87.87% <0%> (-6.07%)	⬇️
clientv3/concurrency/session.go	88.63% <0%> (-4.55%)	⬇️
rafthttp/peer.go	90.22% <0%> (-3.01%)	⬇️
etcdserver/api/v3election/election.go	64.7% <0%> (-2.95%)	⬇️
... and 14 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1e9ba31...87d1e99. Read the comment docs.

[gyuho]

@spzala I separated as a function (let's keep them in check.go for now).

Can you create a follow-up PR to use this in other commands?

Thanks!

[spzala]

@gyuho yup, that's perfect. Thanks!!
lgtm

[raoofm]

@gyuho is there a plan to backport to 3.1 and 3.2, if not can we?

[danbeaulieu]

@gyuho Is this expected?

# docker exec 4a5ce770e887 sh -c 'ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key --endpoints=127.0.0.1:2379 version'
etcdctl version: 3.3.9
API version: 3.3

# docker exec 4a5ce770e887 sh -c 'ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key --endpoints=127.0.0.1:2379 check perf --auto-compact --auto-defrag'
Error: unknown flag: --auto-compact

docker exec 4a5ce770e887 sh -c 'ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key --endpoints=127.0.0.1:2379 check perf  --auto-defrag'
Error: unknown flag: --auto-defrag

[dreambo8563]

@danbeaulieu I got the same error, can not use flag --auto-compact and --auto-defrag.
Have you found any solutions?