14When an arbitrary number of hidden layers is allowed, one can devise many alternative schemes.
For instance, a squaring function x → x2 can be approximated on a small interval via linear
combinations of an arbitrary nonlinear function , since a Taylor expansion around a constant c gives
x2 = 2[
(c + x) -
(c) - x
′(c)]∕
′′(c) + O(x3). The only provision here is that
is at least three times
differentiable (or at least four times differentiable if we would have used the more accurate alternative
x2 = [
(c + x) - 2
(c) +
(c-x)]∕
′′(c) + O(x4)). These requirements are satisfied by our C∞ functions
0,
1
and
2. A multiplication xy can subsequently be constructed as a linear combination of squaring functions
through xy =
[(x + y)2 - (x-y)2], xy =
[(x + y)2 -x2 -y2] or xy = -
[(x-y)2 -x2 -y2]. A combination of
additions and multiplications can then be used to construct any multidimensional polynomial, which in turn can
be used to approximate any continuous multidimensional function up to arbitrary accuracy. See also
[33].