Get the latest tech news
Derivatives, Gradients, Jacobians and Hessians
This article explains how these four things fit together and shows some examples of what they are used for. Derivatives Derivatives are the most fundamental concept in calculus. If you have a funct…
Gradient descent simulates a ball rolling down hill to find the lowest point that we can, adjusting step size, and even adding momentum to try and not get stuck in places that are not the true minimum. Since Jacobians describe warping of space, they are also useful in computer graphics, where for instance, you might want to use alpha transparency to fade an object out over a specific number of pixels to perform anti aliasing, but the object may be described in polar coordinates, or be warped in way that makes it hard to know how many units to fade out over in that modified space. Calculating the Hessian can be quite costly both computationally and in regards to how much memory it uses, for machine learning problems that have millions of parameters or more.
Or read this on Hacker News
