Replication of Diffusion Results

[stefan-baumann]

Hi, I'm trying to replicate your results for applying SaShiMi in a diffusion context, and have run into some questions about implementation details along the way. It'd be awesome if you could help me out with them.

1. I have found the diffusion version of the SaShiMi model at https://github.com/HazyResearch/state-spaces/blob/diffwave/sashimi/sashimi.py. I assume that one is the reference implementation. If yes, what parameters did you use? Just bidirectional=True, unet=True, diffwave=True and set the rest to the values specified in Appendix C.2.2 of the paper and their respective default values?
2. In the original model, you use mu-law quantization for the model. Is this something you also use with the diffusion implementation? And are you using an embedding encoder & sequence decoder like for the AR model? If so, how are you implementing this setup, also in regards to e.g. the additive noise?

Best,
Stefan

[albertfgu]

Hi Stefan,

1. Yes, those should be the settings to use.
2. I believe that diffusion models in general don't use quantization and directly model the real values instead.

The setup we used cloned a public implementation of DiffWave and dropped in the Sashimi model. We originally did not release the full model because we wanted to integrate diffusion into this codebase. This ended up not happening due to time limitations and will likely not happen. As part of the ongoing v3 release effort which is scheduled for end of this month, I will probably just create a public fork of that repo with our changes; it will be messy research code but should help reproducibility

[stefan-baumann]

Hi Albert,
Thank you very much for the incredibly quick help once again!
I already guessed that you were basing your implementation of that repo judging from the references in the reused code. It would be awesome if you could release a fork with your changes as you mention that can be used to replicate your results, especially as I have also been facing some issues stemming from the upsampling method in SaShiMi in my reproduction attempts.

[albertfgu]

My Diffwave implementation is released at https://github.com/albertfgu/diffwave-sashimi. It took longer than planned since I ended up improving the infra to make it easier to train new models, beyond just reproducing results.

[stefan-baumann]

Thank you very much! Doubly so for putting in all of the effort to improve it further to make working on it easier!