Mistake in simple automatic differentiation computation?

[Metronome]

Suppose we have an extremely simple program...

myFunction(x)
[imath]\ \ \ \ [/imath]myExpression = 2cos(2x) - sin(4x)
[imath]\ \ \ \ [/imath]return myExpression

...and want to apply automatic differentiation to it at the input [imath]x = \frac{\pi}{2}[/imath]. Since there is only a single line of any substance, this is essentially equivalent to differentiating [imath]f(x) = 2\cos(2x) - \sin(4x)[/imath]. I have higher aspirations to compute programs of multiple lines and multiple variables, but I seem to be making a mistake even in the basic case.

A dual number function, as I understand it in a way that will generalize to the multivariate case (perhaps no longer called a dual number), is the sum of a function and its total differential, all evaluated at a particular point of which each coordinate is also a dual number. So what we want to find is...[math](f(x) + df)|_{x = x_0 + dx}[/math]...which in this problem is [imath](f(x) + f'(x)\epsilon)|_{x = x_0 + \epsilon} = (2\cos(2x) - \sin(4x) - 4\sin(2x)\epsilon - 4\cos(4x)\epsilon)|_{x = \frac{\pi}{2} + \epsilon} = 2\cos(\pi + 2\epsilon) - \sin(2\pi + 4\epsilon) - 4\sin(\pi + 2\epsilon)\epsilon - 4\cos(2\pi + 4\epsilon)\epsilon = -2\cos(2\epsilon) - \sin(4\epsilon) + 4\sin(2\epsilon)\epsilon - 4\cos(4\epsilon)\epsilon[/imath].

This is the part where I think I'm messing it up, but I can't see how. There is not any division or fancier operation to contend with, so I reason that I can replace each individual sinusoid with just the portion of its Maclaurin Expansion which is affine in [imath]\epsilon[/imath]; any higher order terms are not going to be rescued from [imath]\epsilon^2[/imath] annihilation in the final answer by just addition, subtraction, and non-fractional multiplication. Thus [imath]\cos(k\epsilon) = 1[/imath] and [imath]\sin(k\epsilon) = k\epsilon[/imath] for any constant [imath]k[/imath]. Then my answer reduces to [imath]-2(1) - (4\epsilon) + 4(2\epsilon)\epsilon - 4(1)\epsilon[/imath] or...[math]-2 - 8\epsilon[/math]
However, the answer should be [imath]-2 - 4\epsilon[/imath], as computed in Mathematica...


Do I just have a mistake in the algebra, or have I reasoned or understood the setup incorrectly?

 

[blamocur]

You want to use [imath]f(x_0)[/imath], not [imath]f(x_0+\epsilon)[/imath]. This is analogous to an approximation formula [imath]f(x_0+\epsilon) \approx f(x_0) + \epsilon f^\prime(x_0)[/imath] -- you don't use [imath]x_0+\epsilon[/imath] in the right-hand side.

 

[fresh_42]

You messed it up using differentials and evaluations all in one equation. That mixes functions, functionals, and values.

Why do you want to evaluate [imath] f+f' [/imath]?

We have [imath] f=M_2\circ \cos\circ M_2 -\sin\circ M_4 [/imath] where [imath] M_a(x)=a\cdot x [/imath] with [imath]dM_a=M_a, [/imath]and
[math]\begin{array}{lll} df&= d(M_2\circ \cos\circ M_2)-d(\sin\circ M_4)\\[6pt] &=dM_2\,d(\cos\circ M_2)-\cos\circ M_4\,dM_4\\[6pt] &=-M_2 \sin\circ M_2 \,dM_2-\cos\circ M_4\,M_4\\[6pt] &=-M_2^2 \sin\circ M_2 - M_4\cos\circ M_4 \end{array}[/math]in case you insist on writing it as differential forms. The ordinary way is
[math] (2\cos(2x)- \sin(4x))'=-2\sin(2x)\cdot 2-\cos(4x)\cdot 4=-4\sin(2x)-4\cos(4x) [/math]or
[math] \left. \dfrac{d}{dx}\right|_{x=\pi/2}(2\cos(2x)- \sin(4x))=-4\sin(\pi)-4\cos(2\pi)=-4 [/math]or
[math]\begin{array}{lll} \left. \dfrac{d}{dx}\right|_{x=\pi/2+\varepsilon}(2\cos(2x)- \sin(4x))&=-4\sin(\pi+2\varepsilon)-4\cos(2\pi+4\varepsilon)\\[18pt] &=-4 \cos(4 \varepsilon) + 4 \sin(2 \varepsilon) \\[12pt] &=-4 + 8 \varepsilon + 32 \varepsilon^2 - \dfrac{16\varepsilon^3}{3}- \dfrac{128\varepsilon^4}{3} + O(\varepsilon^5) \end{array}[/math]
I hope I made no typos.

 

[Metronome]

Got it! So yeah my setup was wrong. It was what blamocur said about the input not being a dual number, except that's how to go down the analytic solution path of regular calculus. For the automatic differentiation path it's the opposite; I should have kept the dual number structure of the input but not the function, and then it turns into just tracing code execution. What I did was merge them into automalytic differentiation (which doesn't exist but still outputs dual numbers as answers)!