Nan loss for quantization

[syorami]

I'm running the code for 8-bit quantization but found that the training loss always gets NAN while I didn't make a slight modification to the original code. Wondering why this could happen and hoping for your clarification.

[stgzr]

Same issue, using PyTorch 1.1.0

[stgzr]

Same issue, using PyTorch 1.1.0

Change PyTorch version to 1.0.1 may solve the NaN loss issue.

[syorami]

@stgzr Thanks! I would have a try. And this seems strange to me. What's the difference between Pytorch 1.1.0 and 1.0.1 that could lead to this Nan loss?

[chenhongluo]

The same problem, have you solved it? Finding help.

[TonojiKiobya]

Try changing the learning rate, decrease it or increase it by factor of 10 and see