I trained zipformer using 70k hours corpus , base_lr= 0.03 , A100 8GPUS , max_len=1000 , from 30 epochs , the loss diverged: <img width="1920" height="1142" alt="Image" src="https://github.com/user-attachments/assets/00a54f49-fa50-488c-be2a-ded173542238" /> hope to give me some advices about this phenomenon ,thank you very much.
I trained zipformer using 70k hours corpus , base_lr= 0.03 , A100 8GPUS , max_len=1000 , from 30 epochs , the loss diverged:
hope to give me some advices about this phenomenon ,thank you very much.