lucidrains
diff --git a/‎README.md‎
Lines changed: 2 additions & 0 deletions b/‎README.md‎
Lines changed: 2 additions & 0 deletions
diff --git a/‎grokfast.png‎
142 KB b/‎grokfast.png‎
142 KB
@@ -1,3 +1,5 @@
+<img src="./grokfast.png" width="400px"></img>
+
 ## Grokfast - Pytorch (wip)
 
 Explorations into <a href="https://arxiv.org/html/2405.20233v2">"Grokfast, Accelerated Grokking by Amplifying Slow Gradients"</a>, out of Seoul National University in Korea. In particular, will compare it with NAdam on modular addition as well as a few other tasks, since I am curious why those experiments are left out of the paper. If it holds up, will polish it up into a nice package for quick use.
Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,5 @@`
	`1`	`+<img src="./grokfast.png" width="400px"></img>`
	`2`	`+`
`1`	`3`	`## Grokfast - Pytorch (wip)`
`2`	`4`
`3`	`5`	`Explorations into <a href="https://arxiv.org/html/2405.20233v2">"Grokfast, Accelerated Grokking by Amplifying Slow Gradients"</a>, out of Seoul National University in Korea. In particular, will compare it with NAdam on modular addition as well as a few other tasks, since I am curious why those experiments are left out of the paper. If it holds up, will polish it up into a nice package for quick use.`