RWKV, in easy to read code
Just some commented code you can read to get an understanding of how the newer versions of RWKV work.
Disclaimer: This code was designed for expository purposes, not for training or inference. It purposely trades optimality for readability.
For inference test, you can run python inference.py model_path config_path
You will find various related configs in the configs directory.
For RWKV-7 inference test, download https://huggingface.co/BlinkDL/temp-latest-training-models/resolve/cb5a9646100d64ea548318e99220bf6b5eacbfa9/rwkv-x070-rc4a-172m-pile-20241120-ctx4k.pth?download=true
For RWKV-6 inference test, download https://huggingface.co/RWKV/v6-Finch-PTH/resolve/main/RWKV-Finch-1B6.pth?download=true
For RWKV-5 inference test, download https://huggingface.co/BlinkDL/rwkv-5-world/resolve/main/RWKV-5-World-1B5-v2-20231025-ctx4096.pth?download=true