minibatch size and ADAM optimizer

[bssrdf]

Hi,

I am trying to use ggml to train a small scale model (MNIST size) using ADAM optimizer. I have a question about where to scale the loss function with 1/(minibatch_size). In general, should minibatch_size always be accounted for in the loss function calculation (i.e., the gradient for each param is the average over all minibatch_size samples) ? Sorry, this question may sound naive. Thanks,

Edit: the batch size does not matter as it is the loss function's gradient (which is 1) gets propagated backed. One can always divide the batch size to get loss per data point.