Skip to content

Commit 6d64cfe

Browse files
janbenderjanbender
committed
- added finite differences example
1 parent beb7c02 commit 6d64cfe

2 files changed

Lines changed: 375 additions & 1 deletion

File tree

‎examples/finite_differences.html‎

Lines changed: 373 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,373 @@
1+
<!doctype html>
2+
<html class="no-js" lang="en">
3+
<head>
4+
<meta charset="utf-8">
5+
<meta name="viewport" content="width=device-width, initial-scale=1">
6+
<link rel="stylesheet" href="https://interactivecomputergraphics.github.io/physics-simulation/examples/style.css">
7+
<script type="text/x-mathjax-config">
8+
MathJax.Hub.Config({
9+
extensions: ["tex2jax.js"],
10+
jax: ["input/TeX", "output/HTML-CSS"],
11+
tex2jax: {
12+
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
13+
displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
14+
processEscapes: true
15+
},
16+
"HTML-CSS": { fonts: ["TeX"] }
17+
});
18+
</script>
19+
<script type="text/javascript" async src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.4/MathJax.js"></script>
20+
<script src="https://cdn.plot.ly/plotly-2.5.1.min.js"></script>
21+
<title>Finite difference derivatives</title>
22+
<style>
23+
/* the two plots should share the available width instead of
24+
using their intrinsic (fixed) Plotly size, which overflows
25+
the page's max width */
26+
.plots-grid {
27+
grid-template-columns: 1fr 1fr;
28+
gap: 16px;
29+
}
30+
.plots-grid > div {
31+
min-width: 0; /* allow grid items to shrink below content size */
32+
width: 100%;
33+
}
34+
@media (max-width: 820px) {
35+
.plots-grid { grid-template-columns: 1fr; }
36+
}
37+
</style>
38+
</head>
39+
<body>
40+
41+
<header class="page-header">
42+
<h1>Finite difference derivatives</h1>
43+
</header>
44+
45+
<main>
46+
<!-- Simulation panel: plots + controls -->
47+
<div class="card sim-panel">
48+
<div class="sim-canvas-wrap">
49+
<div class="plots-grid" width="100%">
50+
<div id="plotOutput" style="border:0px solid #000000;border-radius: 0px;background-color:#EEEEEE"></div>
51+
<div id="plotOutput2" style="border:0px solid #000000;border-radius: 0px;background-color:#EEEEEE"></div>
52+
</div>
53+
</div>
54+
</div>
55+
<div class="controls-panel" style="width: 100%;align:center;margin-left:auto;margin-right:auto">
56+
<h3>Controls</h3>
57+
<div class="controls-grid" style="width: 420px;align:left">
58+
<label for="fct">Function</label>
59+
<select onchange="plot.changeFunction()" id="fct" size="1">
60+
<option selected="selected">Smooth polynomial</option>
61+
<option>Oscillatory function</option>
62+
<option>Steep transition</option>
63+
<option>Non-smooth function (kink)</option>
64+
</select>
65+
66+
<label for="x0">Evaluation point x&#8320;</label>
67+
<span class="stat-value" id="x0Display">0.00</span>
68+
69+
<label></label>
70+
<input oninput="plot.reset()" id="x0" type="range" min="-1.5" max="2.2" step="0.01" value="1.3">
71+
72+
<label for="hSlider">Step size h</label>
73+
<span class="stat-value" id="hDisplay">1.0e-1</span>
74+
75+
<label></label>
76+
<input oninput="plot.reset()" id="hSlider" type="range" min="-8" max="0" step="0.05" value="-1">
77+
78+
<label for="showForward">Show forward difference</label>
79+
<input onchange="plot.reset()" id="showForward" type="checkbox" checked>
80+
81+
<label for="showBackward">Show backward difference</label>
82+
<input onchange="plot.reset()" id="showBackward" type="checkbox">
83+
84+
<label for="showCentral">Show central difference</label>
85+
<input onchange="plot.reset()" id="showCentral" type="checkbox" checked>
86+
</div>
87+
</div>
88+
89+
<!-- Theory section -->
90+
<div class="card theory">
91+
<h2>Approximating derivatives with finite differences</h2>
92+
<p>Whenever the derivative $f'(x)$ of a function is not available in closed form (or is expensive to derive/implement), it can be approximated numerically from function values alone. The idea is to replace the derivative &mdash; the slope of the <em>tangent</em> &mdash; by the slope of a nearby <em>secant</em> through points that are a small distance $h$ apart. The left plot below shows the function together with the exact tangent and the secants that correspond to each formula. The right plot shows how the approximation error behaves as $h \to 0$.</p>
93+
94+
<h3>Forward, backward and central differences</h3>
95+
<p>The three basic one-dimensional finite difference formulas are:</p>
96+
$$
97+
\begin{align*}
98+
f'(x) &\approx \frac{f(x+h)-f(x)}{h} &&\text{(forward difference)}\\[4pt]
99+
f'(x) &\approx \frac{f(x)-f(x-h)}{h} &&\text{(backward difference)}\\[4pt]
100+
f'(x) &\approx \frac{f(x+h)-f(x-h)}{2h} &&\text{(central difference)}
101+
\end{align*}
102+
$$
103+
104+
<h3>Truncation error: why central differences are more accurate</h3>
105+
<p>The accuracy of each formula follows directly from a Taylor expansion of $f$ around $x$:</p>
106+
$$
107+
\begin{align*}
108+
f(x+h) &= f(x) + h f'(x) + \tfrac{h^2}{2} f''(x) + \tfrac{h^3}{6} f'''(x) + O(h^4)\\
109+
f(x-h) &= f(x) - h f'(x) + \tfrac{h^2}{2} f''(x) - \tfrac{h^3}{6} f'''(x) + O(h^4)
110+
\end{align*}
111+
$$
112+
<p>Inserting the first expansion into the forward difference formula leaves an error term proportional to $h$:</p>
113+
$$\frac{f(x+h)-f(x)}{h} - f'(x) = \tfrac{h}{2} f''(x) + O(h^2) \;\Rightarrow\; \text{error} = O(h).$$
114+
<p>The odd-order terms cancel when the two expansions are subtracted for the central difference, leaving an error term proportional to $h^2$ &mdash; one order more accurate for (almost) the same cost:</p>
115+
$$\frac{f(x+h)-f(x-h)}{2h} - f'(x) = \tfrac{h^2}{6} f'''(x) + O(h^4) \;\Rightarrow\; \text{error} = O(h^2).$$
116+
117+
<h3>Rounding error: why $h$ cannot be made arbitrarily small</h3>
118+
<p>The formulas above suggest choosing $h$ as small as possible. In floating point arithmetic this is not true: $f(x+h)$ and $f(x)$ are nearly equal for small $h$, so their subtraction suffers from <em>catastrophic cancellation</em>. With machine epsilon $\varepsilon \approx 2.2\times10^{-16}$ (IEEE double precision), the rounding error of a difference quotient grows like $\varepsilon/h$, so the <b>total</b> error is roughly</p>
119+
$$\text{error}_\text{forward}(h) \approx \tfrac{h}{2}|f''(x)| + \tfrac{2\varepsilon}{h}|f(x)|, \qquad \text{error}_\text{central}(h) \approx \tfrac{h^2}{6}|f'''(x)| + \tfrac{\varepsilon}{h}|f(x)|.$$
120+
<p>Both expressions have a minimum at an optimal step size: $h^\ast_\text{forward} \approx \sqrt{\varepsilon} \approx 1.5\times10^{-8}$ and $h^\ast_\text{central} \approx \varepsilon^{1/3} \approx 6\times10^{-6}$ (for values of $f$, $f''$, $f'''$ of order 1). This is exactly the characteristic V-shaped curve visible in the right plot: the error first drops with slope $1$ (forward) or $2$ (central) on the log-log plot, then turns around and grows again once rounding error takes over. Try dragging the step size slider all the way to the left to see the curve pass through this minimum.</p>
121+
122+
<h3>Some functions are harder than others</h3>
123+
<p>The four test functions illustrate different failure modes:</p>
124+
<ul>
125+
<li><b>Smooth polynomial</b> $f(x)=x^3-2x^2+x+1$: a well-behaved baseline. Truncation error follows the theoretical $O(h)$ / $O(h^2)$ slopes closely over many orders of magnitude before rounding error takes over.</li>
126+
<li><b>Oscillatory function</b> $f(x)=\sin(15x)$: higher derivatives are large ($|f'''| \le 15^3$), which inflates the constant in front of $h$ resp. $h^2$ in the truncation error. The whole error curve is shifted upward, so a <em>much smaller</em> $h$ is needed to reach the same accuracy as for the polynomial &mdash; and if $h$ is chosen too large relative to the oscillation period, the difference quotient can even get the sign of the slope wrong.</li>
127+
<li><b>Steep transition</b> $f(x)=\tanh(25x)$: away from $x=0$ the function is almost flat and easy, but inside the narrow transition region the second and third derivatives are huge, so $h$ must be small <em>relative to the width of the transition</em>, not just small in absolute terms. Move $x_0$ towards $0$ and watch the error curve shift upward.</li>
128+
<li><b>Non-smooth function (kink)</b> $f(x)=|x|$: at $x_0=0$ the derivative simply does not exist. No choice of $h$ helps here &mdash; forward difference always returns $+1$, backward difference always returns $-1$, and central difference always returns $0$, independent of $h$. This is a fundamentally different failure than truncation or rounding error: finite differences silently produce a number even where no derivative exists, which is an important pitfall to know about.</li>
129+
</ul>
130+
131+
<h3>References</h3>
132+
<ul>
133+
<li>[LeV07] Randall J. LeVeque. Finite Difference Methods for Ordinary and Partial Differential Equations. SIAM, 2007.</li>
134+
</ul>
135+
</div>
136+
137+
</main>
138+
139+
<script id="simulation_code" type="text/javascript">
140+
class Plot
141+
{
142+
constructor()
143+
{
144+
// meta data (view range, default evaluation point) for every test function
145+
this.functions = {
146+
"Smooth polynomial": {
147+
f: x => x*x*x - 2*x*x + x + 1,
148+
df: x => 3*x*x - 4*x + 1,
149+
xMin: -1.5, xMax: 2.2, x0: 1.3
150+
},
151+
"Oscillatory function": {
152+
f: x => Math.sin(15*x),
153+
df: x => 15*Math.cos(15*x),
154+
xMin: -0.7, xMax: 0.7, x0: 0.25
155+
},
156+
"Steep transition": {
157+
f: x => Math.tanh(25*x),
158+
df: x => 25*(1 - Math.tanh(25*x)*Math.tanh(25*x)),
159+
xMin: -0.3, xMax: 0.3, x0: 0.03
160+
},
161+
"Non-smooth function (kink)": {
162+
f: x => Math.abs(x),
163+
df: x => x === 0 ? NaN : Math.sign(x),
164+
xMin: -1.0, xMax: 1.0, x0: 0.0
165+
}
166+
};
167+
168+
this.changeFunction();
169+
}
170+
171+
// called when the user picks a different function: adjusts the
172+
// x0 slider range to a sensible window for that function and resets
173+
changeFunction()
174+
{
175+
let name = document.getElementById('fct').value;
176+
let info = this.functions[name];
177+
let x0Input = document.getElementById('x0');
178+
x0Input.min = info.xMin;
179+
x0Input.max = info.xMax;
180+
x0Input.step = (info.xMax - info.xMin) / 400;
181+
x0Input.value = info.x0;
182+
this.reset();
183+
}
184+
185+
forwardDiff(f, x, h) { return (f(x+h) - f(x)) / h; }
186+
backwardDiff(f, x, h) { return (f(x) - f(x-h)) / h; }
187+
centralDiff(f, x, h) { return (f(x+h) - f(x-h)) / (2*h); }
188+
189+
reset()
190+
{
191+
this.name = document.getElementById('fct').value;
192+
this.info = this.functions[this.name];
193+
this.x0 = parseFloat(document.getElementById('x0').value);
194+
this.logH = parseFloat(document.getElementById('hSlider').value);
195+
this.h = Math.pow(10, this.logH);
196+
this.showForward = document.getElementById('showForward').checked;
197+
this.showBackward = document.getElementById('showBackward').checked;
198+
this.showCentral = document.getElementById('showCentral').checked;
199+
200+
document.getElementById('x0Display').innerHTML = this.x0.toFixed(3);
201+
document.getElementById('hDisplay').innerHTML = this.h.toExponential(2);
202+
203+
this.plotFunction();
204+
this.plotConvergence();
205+
}
206+
207+
// left plot: function, exact tangent and the finite difference secants
208+
plotFunction()
209+
{
210+
let info = this.info;
211+
let x0 = this.x0;
212+
let h = this.h;
213+
let f = info.f;
214+
215+
let num = 400;
216+
let xs = [], ys = [];
217+
for (let i = 0; i <= num; i++)
218+
{
219+
let x = info.xMin + (info.xMax - info.xMin) * i / num;
220+
xs.push(x);
221+
ys.push(f(x));
222+
}
223+
224+
let f_x0 = f(x0);
225+
let df_exact = info.df(x0);
226+
227+
// tangent line has a fixed, human-visible length independent of h
228+
let half = (info.xMax - info.xMin) * 0.12;
229+
let tangent = {
230+
x: [x0 - half, x0 + half],
231+
y: isNaN(df_exact) ? [null, null] : [f_x0 - df_exact*half, f_x0 + df_exact*half],
232+
name: "tangent (exact f'(x₀))",
233+
mode: 'lines',
234+
line: { color: 'rgb(0,150,0)', width: 3 }
235+
};
236+
237+
let f_xph = f(x0 + h);
238+
let f_xmh = f(x0 - h);
239+
240+
let forward = {
241+
x: [x0, x0 + h], y: [f_x0, f_xph],
242+
name: 'forward difference', mode: 'lines+markers',
243+
line: { color: 'rgb(230,120,0)', dash: 'dash', width: 2 }
244+
};
245+
let backward = {
246+
x: [x0 - h, x0], y: [f_xmh, f_x0],
247+
name: 'backward difference', mode: 'lines+markers',
248+
line: { color: 'rgb(180,0,180)', dash: 'dot', width: 2 }
249+
};
250+
let central = {
251+
x: [x0 - h, x0 + h], y: [f_xmh, f_xph],
252+
name: 'central difference', mode: 'lines+markers',
253+
line: { color: 'rgb(0,90,220)', dash: 'dashdot', width: 2 }
254+
};
255+
256+
let marker = {
257+
x: [x0], y: [f_x0], mode: 'markers+text',
258+
text: ['x₀'], textposition: 'top center',
259+
marker: { color: 'black', size: 8 },
260+
name: 'x₀', showlegend: false
261+
};
262+
263+
let data = [
264+
{ x: xs, y: ys, name: this.name, line: { color: 'rgb(90,90,90)' } },
265+
tangent
266+
];
267+
if (this.showForward) data.push(forward);
268+
if (this.showBackward) data.push(backward);
269+
if (this.showCentral) data.push(central);
270+
data.push(marker);
271+
272+
let annotations = [];
273+
if (isNaN(df_exact))
274+
{
275+
annotations.push({
276+
x: x0, y: f_x0, xref: 'x', yref: 'y',
277+
text: "f is not differentiable at x₀!",
278+
showarrow: true, arrowhead: 4, ax: 60, ay: -40
279+
});
280+
}
281+
282+
var layout = {
283+
title: 'Function and finite difference secants',
284+
autosize: true, height: 480,
285+
margin: { t: 40, r: 20, b: 40, l: 50 },
286+
xaxis: { range: [info.xMin, info.xMax] },
287+
annotations: annotations
288+
};
289+
290+
Plotly.newPlot('plotOutput', data, layout, { responsive: true });
291+
}
292+
293+
// right plot: log-log convergence plot of the approximation error over h
294+
plotConvergence()
295+
{
296+
let info = this.info;
297+
let x0 = this.x0;
298+
let f = info.f;
299+
let df_exact = info.df(x0);
300+
301+
let hs = [], errF = [], errB = [], errC = [];
302+
let n = 160;
303+
let minFloor = 1.0e-18; // avoid log(0) on the log axis
304+
for (let i = 0; i <= n; i++)
305+
{
306+
let logh = -16 + 16 * i / n; // h from 1e-16 to 1
307+
let h = Math.pow(10, logh);
308+
let fF = this.forwardDiff(f, x0, h);
309+
let fB = this.backwardDiff(f, x0, h);
310+
let fC = this.centralDiff(f, x0, h);
311+
hs.push(h);
312+
errF.push(Math.max(Math.abs(fF - df_exact), minFloor));
313+
errB.push(Math.max(Math.abs(fB - df_exact), minFloor));
314+
errC.push(Math.max(Math.abs(fC - df_exact), minFloor));
315+
}
316+
317+
let data = [];
318+
let annotations = [];
319+
320+
if (isNaN(df_exact))
321+
{
322+
annotations.push({
323+
x: 0.5, y: 0.5, xref: 'paper', yref: 'paper',
324+
text: "f'(x₀) does not exist —<br>no error can be defined here",
325+
showarrow: false, font: { size: 13 }
326+
});
327+
}
328+
else
329+
{
330+
if (this.showForward)
331+
data.push({ x: hs, y: errF, name: '|forward error|', line: { color: 'rgb(230,120,0)' } });
332+
if (this.showBackward)
333+
data.push({ x: hs, y: errB, name: '|backward error|', line: { color: 'rgb(180,0,180)' } });
334+
if (this.showCentral)
335+
data.push({ x: hs, y: errC, name: '|central error|', line: { color: 'rgb(0,90,220)' } });
336+
337+
// reference slopes O(h) and O(h^2), anchored via numerically
338+
// estimated f''(x0) / f'''(x0) so they line up with the actual data
339+
let d = 1.0e-3;
340+
let f2 = (f(x0+d) - 2*f(x0) + f(x0-d)) / (d*d);
341+
let f3 = (f(x0+2*d) - 2*f(x0+d) + 2*f(x0-d) - f(x0-2*d)) / (2*d*d*d);
342+
let refO1 = hs.map(h => Math.max(0.5*Math.abs(f2)*h, minFloor));
343+
let refO2 = hs.map(h => Math.max((1/6)*Math.abs(f3)*h*h, minFloor));
344+
data.push({ x: hs, y: refO1, name: 'O(h) reference', line: { color: 'rgb(150,150,150)', dash: 'dash' } });
345+
data.push({ x: hs, y: refO2, name: 'O(h²) reference', line: { color: 'rgb(150,150,150)', dash: 'dot' } });
346+
}
347+
348+
// marker for the h currently selected by the slider
349+
let shapes = [{
350+
type: 'line', xref: 'x', yref: 'paper',
351+
x0: this.h, x1: this.h, y0: 0, y1: 1,
352+
line: { color: 'rgba(0,0,0,0.4)', width: 1, dash: 'dash' }
353+
}];
354+
355+
var layout = {
356+
title: 'Error vs. step size h',
357+
autosize: true, height: 480,
358+
margin: { t: 40, r: 20, b: 40, l: 60 },
359+
xaxis: { title: 'h', type: 'log', autorange: 'reversed' },
360+
yaxis: { title: '|error|', type: 'log' },
361+
shapes: shapes,
362+
annotations: annotations
363+
};
364+
365+
Plotly.newPlot('plotOutput2', data, layout, { responsive: true });
366+
}
367+
}
368+
369+
plot = new Plot();
370+
</script>
371+
372+
</body>
373+
</html>

‎index.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Physics simulation is an important research topic in visual computing. It has many applications ranging from such as virtual prototyping, training simulators, robotics, animation software for digital production including visual effects in film and animation movies, and computer games – just to mention a few.
44

5-
At [RWTH Aachen university](https://animation.rwth-aachen.de) I offer [lectures](https://animation.rwth-aachen.de/courses/) which give an introduction to state-of-the-art simulation methods for rigid bodies, deformable solids and fluids in the area of visual computing (e.g, the Finite Element Method (FEM) and the [Smoothed Particle Hydrodynamics (SPH)](https://interactivecomputergraphics.github.io/SPH-Tutorial/) approach). For the lectures I programmed several simulation examples using JavaScript.
5+
At [RWTH Aachen university](https://animation.rwth-aachen.de) I offer [lectures](https://animation.rwth-aachen.de/courses/) which give an introduction to state-of-the-art simulation methods for rigid bodies, deformable solids and fluids in the area of visual computing (e.g, the Finite Element Method (FEM) and the [Smoothed Particle Hydrodynamics (SPH)](https://sph-tutorial.physics-simulation.org) approach). For the lectures I programmed several simulation examples using JavaScript.
66

77
Why JavaScript? Because everybody can directly run the examples in the browser. Moreover, I was able to add a short web page for each example which explains the method.
88

@@ -11,6 +11,7 @@ I hope the examples and their documentation help you to learn something about th
1111
# General
1212

1313
* [Barycentric coordinates](examples/barycentric_coordinates.html)
14+
* [Finite differences](examples/finite_differences.html)
1415
* [Heat equation](examples/heat_equation.html)
1516

1617
# Newton's method

0 commit comments

Comments
 (0)