It's been established that the latter point is wrong arxiv.org/pdf/2506.14965, improving math reasoning only helps upto a certain point
And it's also obvious that the primary reason behind focusing on research math is publicity, not the need to improve some non-existent transferability in reasoning skills
Someone took a pic of what I鈥檝e been doing for the past few days
PyTorch can do a lot of sneaky down/upcasting that messes with your training and memory efficiency