Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoder: move encoder to a independent mod #5

Merged
merged 1 commit into from
Jan 12, 2022
Merged

Conversation

3pointer
Copy link
Owner

@3pointer 3pointer commented Dec 29, 2021

As title

Copy link
Collaborator

@YuJuncen YuJuncen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM
(BTW, we can wrap the interface over Read or Write interface then)

components/br-stream/src/codec.rs Outdated Show resolved Hide resolved
let len = buf.get_u32_be() as usize;
let mut buffer = vec![0; len];
buf.read_exact(buffer.as_mut_slice()).unwrap();
let val = buffer.to_vec();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems buffer is already Vec? (This to_vec may introduce unnecessary copying)

@YuJuncen YuJuncen merged commit 9d697c4 into br-stream Jan 12, 2022
YuJuncen added a commit that referenced this pull request Apr 26, 2022
…h `br-stream`:

- backup-stream: update kvproto to master [by @3pointer] (#48)

- Added retry for initial scanning and some metrics [by @YuJuncen] (#47)

- Added a new observer type 'pitr' [by @YuJuncen] (#46)

- backup-stream: fix some bugs in log backup [by @3pointer] (#45)

- Refactor resolver [by @YuJuncen] (#44)

- report error to PD server [by @YuJuncen] (#43)

- br: support pause/resume stream task [by @joccau] (#42)

- Adapt error code for `endpoint::Error` and implement the contextual error [by @YuJuncen] (#41)

- set uuid for header to prevent raftstore merging [by @YuJuncen] (#40)

- fix the upload part of S3 storage [by @YuJuncen] (#39)

- Use min ts of mem lock [by @YuJuncen] (#38)

- Fix size leakage and build [by @YuJuncen] (#37)

- eliminate the block call in ticker [by @YuJuncen] (#36)

- br: support checkSum during stream restore dml kv-events [by @joccau] (#35)

- refine br-stream to backup-stream [by @3pointer] (#34)

- Allow local storage support directory and partition the log files by table [by @YuJuncen] (#33)

- br-stream: added store error to the store [by @YuJuncen] (#32)

- Scan on Leader Change [by @YuJuncen] (#31)

- use local thread pool for downloading [by @YuJuncen] (#30)

- display the error when failed to get snapshot [by @YuJuncen] (#29)

- br-stream: remove duplicate entry in apply kv file [by @3pointer] (#28)

- don't hold lock during flushing [by @YuJuncen] (#27)

- fix listener on follower if region changed [by @YuJuncen] (#26)

- Update Service GC Safe Point after Flushing [by @YuJuncen] (#25)

- br-stream: use raft router to apply kv files for sst_importer [by @3pointer] (#24)

- added integration test framework [by @YuJuncen] (#22)

- br-stream: add restore ts to filter data out of range. [by @3pointer] (#20)

- *: batch write to temp file [by @YuJuncen] (#19)

- omit coping in EventIterator, use write batch for apply [by @YuJuncen] (#18)

- br: don't flush to externStorage periodically when have empty kv-record [by @joccau] (#17)

- added SegmentMap to replace the BTreeMap [by @YuJuncen] (#16)

- added resolved ts related metrics [by @YuJuncen] (#15)

- Added resolved timestamp uploading [by @YuJuncen] (#14)

- add flush tick [by @3pointer] (#13)

- implement stream restore for tikv side. [by @3pointer] (#12)

- added metrics [by @YuJuncen] (#11)

- update kvproto [by @YuJuncen] (#9)

- Initial scanning && Error reporting [by @YuJuncen] (#8)

- resolve conflict [by @3pointer] (#7)

- br: backup stream: support flushing temp files to ExternalStorage [by @joccau] (#6)

- encoder: move encoder to a independent mod [by @3pointer] (#5)

- br-stream: don't clone the key & value in encode_event() [by @kennytm] (#4)

- br: backup stream: Modify log print format [by @joccau] (#3)

- br-stream: reduce the lock of tables; added some metrics [by @YuJuncen] (#2)

For more details of these commits, please check the origin feature branch at https://github.com/3pointer/tikv/tree/br-stream.

Signed-off-by: Yu Juncen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants