Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can you write some docs or notes to explain the content of the data pre-processing,such as fucn: _process_list ? #88

Open
imoisture opened this issue Jun 27, 2023 · 0 comments

Comments

@imoisture
Copy link

imoisture commented Jun 27, 2023

数据预处理的步骤似乎有一点复杂。
我阅读了一段时间的代码之后,还是不是很明白有些预处理的代码目的是做什么?
作者能否写个文档或者注释,阐述一下一些数据预处理相关函数的具体左右,比如convert_data_to_id, _convert_to_tensors

有些变量的含义也不是太理解,比如 input_id_subs,ext_tables,segment_rel,segment_rel_offset 🌝

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant