Gradient and Maximum Increase of a Function
Date: 07/19/2005 at 04:58:42 From: Jill Subject: Linear Algebra and Calculus I am having difficulty explaining why the gradient points in the direction of the maximum increase of a function. Several resources make the statement, but no one explains it. It is probably something simple I am overlooking. Can the dot product be used to justify it? Thanks.
Date: 07/19/2005 at 05:54:56 From: Doctor Jerry Subject: Re: Linear Algebra and Calculus Hello Jill, One explanantion uses the idea of "directional derivative," which usually precedes the definition of the gradient. If f is a function of two variables, if a = <a1,a2> is a point (specified, for convenience, as a position vector) in the domain, and if u = <u1,u2> is a unit vector (thought of as being based at a), then the directional derivative of f at a and in the direction u is the limit of the ratio [ f(a+h*u) - f(a) ] / h as h->0. If this limit exists it is often denoted as D_u f(a), where "_" means subscript. This is the rate of change of f in the direction u. It is easy to see that if u = <1,0> and <0,1>, the directional derivatives are the partials f_x(a) and f_y(a) of f at a, respectively. If f is differentiable at a, one can show that D_u f(a) = <f_x(a), f_y(a)> dot u. (dot = dot product of vectors) Of course, <f_x(a), f_y(a)> is the gradient of f at a. Recalling that the dot product of two vectors is the product of their lengths and the cosine of the angle between them, it follows that the maximum directional derivative happens when u is the direction of the gradient, that is, the cosine of the angle between them is 0. Please write back if my comments are not clear. - Doctor Jerry, The Math Forum http://mathforum.org/dr.math/
Date: 07/19/2005 at 11:48:09 From: Doctor George Subject: Re: Linear Algebra and Calculus Hi Jill, Doctor Jerry gave a very nice answer to your question. Here is another approach to it. Using Doctor Jerry's notation, if we examine the Taylor expansion of the function, the linear term is h <f_x(a), f_y(a)> dot u For sufficiently small values of "h" the linear term will dominate the expansion, so the maximum increase in a small neighborhood about "a" will occur when the linear term is maximized. By Doctor Jerry's reasoning, this happens when "u" is the direction of the gradient. Just to broaden the picture a bit, in optimization theory the goal is often the minimization of a function. The direction opposite the gradient is called the direction of "steepest descent." - Doctor George, The Math Forum http://mathforum.org/dr.math/
Date: 07/19/2005 at 13:43:29 From: Jill Subject: Thank you (Linear Algebra and Calculus) Thanks a bunch! I appreciate you both taking the time to help!
Search the Dr. Math Library:
Ask Dr. MathTM
© 1994-2015 The Math Forum