We consider infinitely wide multi-layer perceptrons (MLPs) which are limits of standard deep feed-forward neural networks. We assume that, for each layer, the weights of an MLP are initialized with independent and identically distributed (i.i.d.) samples from either a light-tailed (finite-variance) or a heavy-tailed distribution in the domain of attraction of a symmetric $\alpha$-stable distribution, where $\alpha\in(0,2]$ may depend on the layer. For the bias terms of the layer, we assume i.i.d. initializations with a symmetric $\alpha$-stable distribution having the same $\alpha$ parameter as that layer. Non-stable heavy-tailed weight distributions are important since they have been empirically seen to emerge in trained deep neural nets such as the ResNet and VGG series, and proven to naturally arise via stochastic gradient descent. The introduction of heavy-tailed weights broadens the class of priors in Bayesian neural networks. In this work we extend a recent result of Favaro, Fortini, and Peluchetti (2020) to show that the vector of pre-activation values at all nodes of a given hidden layer converges in the limit, under a suitable scaling, to a vector of i.i.d. random variables with symmetric $\alpha$-stable distributions, $\alpha\in(0,2]$.

This paper discusses three methods of training multi-layer perceptrons (MLPs) to model a six-degrees-of- freedom inertial sensor. Such a sensor is designed for use with a robot to determine the location of objects it has to pick up. The sensor operates by measuring parameters related to the inertia of an object and computing its location from those parameters. MLP models are employed for part of the computation. They are trained to output the orientation of the object in response to an input pattern that includes the period of natural vibration of the sensor on which the object rests. After reviewing the working principle of the sensor, the paper describes the three MLP training methods (backpropagation, optimisation using the Levenberg-Marquardt algorithm, evolution based on the genetic algorithm) and presents the experimental results obtained.