Evolving Connectionist Systems: The Knowledge Engineering Approach Book

Foreword I   Walter J. Freeman     vii
Foreword II   John G. Taylor     ix
Preface     xi
Abstract     xxi
Evolving Connectionist Methods     1
Introduction     3
Everything Is Evolving, but What Are the Evolving Rules?     3
Evolving Intelligent Systems (EIS) and Evolving Connectionist Systems (ECOS)     8
Biological Inspirations for EIS and ECOS     11
About the Book     13
Further Reading     13
Feature Selection, Model Creation, and Model Validation     15
Feature Selection and Feature Evaluation     15
Incremental Feature Selection     20
Machine Learning Methods - A Classification Scheme     21
Probability and Information Measure. Bayesian Classifiers, Hidden Markov Models. Multiple Linear Regression     35
Support Vector Machines (SVM)     40
Inductive Versus Transductive Learning and Reasoning. Global, Local, and 'Personalised' Modelling     44
Model Validation     48
Exercise     49
Summary and Open Problems     49
Further Reading     51
Evolving Connectionist Methods for Unsupervised Learning     53
Unsupervised Learningfrom Data. Distance Measure     53
Clustering     57
Evolving Clustering Method (ECM)     61
Vector Quantisation. SOM and ESOM     68
Prototype Learning. ART     73
Generic Applications of Unsupervised Learning Methods     75
Exercise     81
Summary and Open Problems     81
Further Reading     82
Evolving Connectionist Methods for Supervised Learning     83
Connectionist Supervised Learning Methods     83
Simple Evolving Connectionist Methods     91
Evolving Fuzzy Neural Networks (EFuNN)     97
Knowledge Manipulation in Evolving Fuzzy Neural Networks (EFuNNs) - Rule Insertion, Rule Extraction, Rule Aggregation     109
Exercise     124
Summary and Open Questions     125
Further Reading     126
Brain Inspired Evolving Connectionist Models     127
State-Based ANN     127
Reinforcement Learning     132
Evolving Spiking Neural Networks     133
Summary and Open Questions     139
Further Reading     140
Evolving Neuro-Fuzzy Inference Models     141
Knowledge-Based Neural Networks     141
Hybrid Neuro-Fuzzy Inference System (HyFIS)     146
Dynamic Evolving Neuro-Fuzzy Inference Systems (DENFIS)     149
Transductive Neuro-Fuzzy Inference Models     161
Other Evolving Fuzzy Rule-Based Connectionist Systems     168
Exercise     175
Summary and Open Problems     175
Further Reading     175
Population-Generation-Based Methods: Evolutionary Computation     177
A Brief Introduction to EC     177
Genetic Algorithms and Evolutionary Strategies     179
Traditional Use of EC for Learning and Optimisation in ANN     183
EC for Parameter and Feature Optimisation of ECOS     185
EC for Feature and Model Parameter Optimisation of Transductive Personalised (Nearest Neighbour) Models     194
Particle Swarm Intelligence     198
Artificial Life Systems (ALife)     200
Exercise     201
Summary and Open Questions     202
Further Reading     202
Evolving Integrated Multimodel Systems     203
Evolving Multimodel Systems     203
ECOS for Adaptive Incremental Data and Model Integration     209
Integrating Kernel Functions and Regression Formulas in Knowledge-Based ANN     215
Ensemble Learning Methods for ECOS      219
Integrating ECOS and Evolving Ontologies     225
Conclusion and Open Questions     226
Further Reading     227
Evolving Intelligent Systems     229
Adaptive Modelling and Knowledge Discovery in Bioinformatics     231
Bioinformatics: Information Growth, and Emergence of Knowledge     231
DNA and RNA Sequence Data Analysis and Knowledge Discovery     236
Gene Expression Data Analysis, Rule Extraction, and Disease Profiling     242
Clustering of Time-Course Gene Expression Data     259
Protein Structure Prediction     262
Gene Regulatory Networks and the System Biology Approach     265
Summary and Open Problems     272
Further Reading     273
Dynamic Modelling of Brain Functions and Cognitive Processes     275
Evolving Structures and Functions in the Brain and Their Modelling     275
Auditory, Visual, and Olfactory Information Processing and Their Modelling     282
Adaptive Modelling of Brain States Based on EEG and fMRI Data     290
Computational Neuro-Genetic Modelling (CNGM)     295
Brain-Gene Ontology     299
Summary and Open Problems     301
Further Reading     302
Modelling the Emergence of Acoustic Segments in Spoken Languages     303
Introduction to the Issues of Learning Spoken Languages     303
The Dilemma 'Innateness Versus Learning' or 'Nature Versus Nurture' Revisited     305
ECOS for Modelling the Emergence of Phones and Phonemes     307
Modelling Evolving Bilingual Systems     316
Summary and Open Problems     321
Further Reading     323
Evolving Intelligent Systems for Adaptive Speech Recognition     325
Introduction to Adaptive Speech Recognition     325
Speech Signal Analysis and Speech Feature Selection     329
Adaptive Phoneme-Based Speech Recognition     331
Adaptive Whole Word and Phrase Recognition     334
Adaptive, Spoken Language Human-Computer Interfaces     338
Exercise     339
Summary and Open Problems     339
Further Reading     340
Evolving Intelligent Systems for Adaptive Image Processing     341
Image Analysis and Feature Selection     341
Online Colour Quantisation     344
Adaptive Image Classification     348
Incremental Face Membership Authentication and Face Recognition     350
Online Video-Camera Operation Recognition     353
Exercise      357
Summary and Open Problems     358
Further Reading     358
Evolving Intelligent Systems for Adaptive Multimodal Information Processing     361
Multimodal Information Processing     361
Adaptive, Integrated, Auditory and Visual Information Processing     362
Adaptive Person Identification Based on Integrated Auditory and Visual Information     364
Person Verification Based on Auditory and Visual Information     373
Summary and Open Problems     379
Further Reading     380
Evolving Intelligent Systems for Robotics and Decision Support     381
Adaptive Learning Robots     381
Modelling of Evolving Financial and Socioeconomic Processes     382
Adaptive Environmental Risk of Event Evaluation     385
Summary and Open Questions     390
Further Reading     391
What Is Next: Quantum Inspired Evolving Intelligent Systems?     393
Why Quantum Inspired EIS?     393
Quantum Information Processing     394
Quantum Inspired Evolutionary Optimisation Techniques     396
Quantum Inspired Connectionist Systems     398
Linking Quantum to Neuro-Genetic Information Processing: Is This The Challenge For the Future?     400
Summary and Open Questions     402
Further Reading     403
A Sample Program in MATLAB for Time-Series Analysis     405
A Sample MATLAB Program to Record Speech and to Transform It into FFT Coefficients as Features     407
A Sample MATLAB Program for Image Analysis and Feature Extraction     411
Macroeconomic Data Used in Section 14.2 (Chapter 14)     415
References     417
Extended Glossary     439
Index     453