CHAPTER 2
PARTICLE POTENTIAL FIELDS
The idea of self-organization in distributed systems through pair-wise interactions
of particles emitting potential energies has its origin in the work done by Principe et al.
[PriOO] on the Information Particle Interaction Model (IPIM). IPIM gives a physical
interpretation of the process for finding Renyi's quadratic entropy by which any learning
process can be seen as an interaction of the samples, considered as particles interacting in
a potential field. IPIM has been successfully applied in many problems including
independent component analysis (ICA), nonlinear principal component analysis
(nonlinear PCA) and SAR image feature extraction. Generalized Information Particle
Interaction Model (GIPIM) is the generalization of IPIM, primarily developed for
learning and system adaptation [Erd02]. Self-organization algorithm can be derived as a
special case in GIPIM, corresponding to a specific choice of particle potential function
(elaborated later).
2.1 Generalized Information Particle Interaction Model
The quest by Principe et al. [PriOO] for finding better cost functions than mean
square error (MSE) led to the use of Renyi's quadratic entropy as an adaptation criterion
in optimization problems. This inspired the emergence of a dynamic new branch in
adaptive systems, termed Information Theoretic Learning (ITL). In ITL, Renyi's
quadratic entropy was combined with nonparametric estimation of probability density
functions (pdf) using Parzen windows, to develop a highly effective algorithm for
computing entropy as described below.