Language models recognize dropout and Gaussian noise applied to their activations
3
뭔가 activation monitoring을 하고 있으면 GPU 연산에 변화가 있고 그걸 LLM이 감지하게 훈련할 수 있다 같은 논문을 트위터에서 봤었는데 이게 먼저인건가?
openreview.net/forum?id=uiDb…
How do we understand neural nets with so many interacting params that we can’t describe what they do in words?
Our take: objective functions and regularizers offer a concise description of "what models learn"
Can we *empirically* study implicit regularization in DL?🧵