Associated Topics || Dr. Math Home || Search Dr. Math

### Gradient and Maximum Increase of a Function

```Date: 07/19/2005 at 04:58:42
From: Jill
Subject: Linear Algebra and Calculus

I am having difficulty explaining why the gradient points in the
direction of the maximum increase of a function.  Several resources
make the statement, but no one explains it.  It is probably something
simple I am overlooking.  Can the dot product be used to justify it?

Thanks.

```

```
Date: 07/19/2005 at 05:54:56
From: Doctor Jerry
Subject: Re: Linear Algebra and Calculus

Hello Jill,

One explanantion uses the idea of "directional derivative," which
usually precedes the definition of the gradient.

If f is a function of two variables, if a = <a1,a2> is a point
(specified, for convenience, as a position vector) in the domain, and
if u = <u1,u2> is a unit vector (thought of as being based at a), then
the directional derivative of f at a and in the direction u is the
limit of the ratio

[ f(a+h*u) - f(a) ] / h

as h->0.  If this limit exists it is often denoted as D_u f(a), where
"_" means subscript.  This is the rate of change of f in the direction
u.  It is easy to see that if u = <1,0> and <0,1>, the directional
derivatives are the partials f_x(a) and f_y(a) of f at a,
respectively.

If f is differentiable at a, one can show that

D_u f(a) = <f_x(a), f_y(a)> dot u. (dot = dot product of vectors)

Of course, <f_x(a), f_y(a)> is the gradient of f at a.  Recalling that
the dot product of two vectors is the product of their lengths and the
cosine of the angle between them, it follows that the maximum
directional derivative happens when u is the direction of the
gradient, that is, the cosine of the angle between them is 0.

Please write back if my comments are not clear.

- Doctor Jerry, The Math Forum
http://mathforum.org/dr.math/

```

```
Date: 07/19/2005 at 11:48:09
From: Doctor George
Subject: Re: Linear Algebra and Calculus

Hi Jill,

Doctor Jerry gave a very nice answer to your question.  Here is
another approach to it.

Using Doctor Jerry's notation, if we examine the Taylor expansion of
the function, the linear term is

h <f_x(a), f_y(a)> dot u

For sufficiently small values of "h" the linear term will dominate the
expansion, so the maximum increase in a small neighborhood about "a"
will occur when the linear term is maximized.  By Doctor Jerry's
reasoning, this happens when "u" is the direction of the gradient.

Just to broaden the picture a bit, in optimization theory the goal is
often the minimization of a function.  The direction opposite the
gradient is called the direction of "steepest descent."

- Doctor George, The Math Forum
http://mathforum.org/dr.math/

```

```
Date: 07/19/2005 at 13:43:29
From: Jill
Subject: Thank you (Linear Algebra and Calculus)

Thanks a bunch!  I appreciate you both taking the time to help!
```
Associated Topics:
College Calculus
College Linear Algebra

Search the Dr. Math Library:

 Find items containing (put spaces between keywords):   Click only once for faster results: [ Choose "whole words" when searching for a word like age.] all keywords, in any order at least one, that exact phrase parts of words whole words

Submit your own question to Dr. Math
Math Forum Home || Math Library || Quick Reference || Math Forum Search