Skip to content

SmerkyG/RWKV_Explained

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

28 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RWKV_Explained

RWKV, in easy to read code

Just some commented code you can read to get an understanding of how the newer versions of RWKV work.

Disclaimer: This code was designed for expository purposes, not for training or inference. It purposely trades optimality for readability.

For inference test, you can run python inference.py model_path config_path You will find various related configs in the configs directory. For RWKV-7 inference test, download https://huggingface.co/BlinkDL/temp-latest-training-models/resolve/cb5a9646100d64ea548318e99220bf6b5eacbfa9/rwkv-x070-rc4a-172m-pile-20241120-ctx4k.pth?download=true For RWKV-6 inference test, download https://huggingface.co/RWKV/v6-Finch-PTH/resolve/main/RWKV-Finch-1B6.pth?download=true For RWKV-5 inference test, download https://huggingface.co/BlinkDL/rwkv-5-world/resolve/main/RWKV-5-World-1B5-v2-20231025-ctx4096.pth?download=true

About

RWKV, in easy to read code

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages