CHAPTER 2
 PARTICLE POTENTIAL FIELDS

 The idea of self-organization in distributed systems through pair-wise interactions

of particles emitting potential energies has its origin in the work done by Principe et al.

[PriOO] on the Information Particle Interaction Model (IPIM). IPIM gives a physical

interpretation of the process for finding Renyi's quadratic entropy by which any learning

process can be seen as an interaction of the samples, considered as particles interacting in

a potential field. IPIM has been successfully applied in many problems including

independent component analysis (ICA), nonlinear principal component analysis

(nonlinear PCA) and SAR image feature extraction. Generalized Information Particle

Interaction Model (GIPIM) is the generalization of IPIM, primarily developed for

learning and system adaptation [Erd02]. Self-organization algorithm can be derived as a

special case in GIPIM, corresponding to a specific choice of particle potential function

(elaborated later).

 2.1 Generalized Information Particle Interaction Model

 The quest by Principe et al. [PriOO] for finding better cost functions than mean

square error (MSE) led to the use of Renyi's quadratic entropy as an adaptation criterion

in optimization problems. This inspired the emergence of a dynamic new branch in

adaptive systems, termed Information Theoretic Learning (ITL). In ITL, Renyi's

quadratic entropy was combined with nonparametric estimation of probability density

functions (pdf) using Parzen windows, to develop a highly effective algorithm for

computing entropy as described below.