Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

intuition behind trainable weights and bias for losses #25

Open
DikshaMeghwal opened this issue Apr 15, 2024 · 1 comment
Open

intuition behind trainable weights and bias for losses #25

DikshaMeghwal opened this issue Apr 15, 2024 · 1 comment

Comments

@DikshaMeghwal
Copy link

DikshaMeghwal commented Apr 15, 2024

Hey
I was wondering if you could shed some light on why did you add learnable weights and biases to the sync and identity losses? To me it seemed like you were possibly trying to scale and shift but I don't understand why the model doesn't train without it.
Also if you put learnable weights and biases, whats stopping the weight to be 0 and making loss 0?
I am using voxceleb dataset for training.

Below are the loss curves for when I removed the weights. The sync loss seems to be stagnant while the identity loss is increasing.
Screenshot 2024-04-15 at 12 46 35 PM
Screenshot 2024-04-15 at 12 46 51 PM

@6eternal6
Copy link

hello,
Seeing that you have successfully trained this code, I would like to ask what is the meaning of offset in dataLoader() when processing the data? Is this taken from the txt file?
Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants