We are trying to minimize the depicted function f(x,y) given some initial positions and the local gradient ∇f(x,y). The green circle represents the initial position, blue circles depict ADAM updates, red circles SGD updates. The vector lines coming out of the circles are the gradients. Playing with β₁ and β₂ in ADAM will reveal where the physics-borrowed "momentum" nomenclature comes from.
georggrab.net