Computational Intelligence : Methods and Techniques Book

Foreword v

1 Introduction 1

2 Selected issues of artificial intelligence 7

2.1 Introduction 7

2.2 An outline of artificial intelligence history 8

2.3 Expert systems 10

2.4 Robotics 11

2.5 Processing of speech and natural language 13

2.6 Heuristics and research strategies 15

2.7 Cognitivistics 16

2.8 Intelligence of ants 17

2.9 Artificial life 19

2.10 Bots 20

2.11 Perspectives of artificial intelligence development 22

2.12 Notes 23

3 Methods of knowledge representation using rough sets 25

3.1 Introduction 25

3.2 Basic terms 27

3.3 Set approximation 34

3.4 Approximation of family of sets 44

3.5 Analysis of decision tables 46

3.6 Application of LERS software 54

3.7 Notes 61

4 Methods of knowledge representation using type-1 fuzzy sets 63

4.1 Introduction 63

4.2 Basic terms and definitions of fuzzy sets theory 63

4.3 Operations on fuzzy sets 76

4.4 The extension principle 83

4.5 Fuzzy numbers 87

4.6 Triangular norms and negations 96

4.7 Fuzzy relations and their properties 108

4.8 Approximate reasoning 112

4.8.1 Basic rules of inference in binary logic 112

4.8.2 Basic rules of inference in fuzzy logic 114

4.8.3 Inference rules for the Mamdani model 118

4.8.4 Inference rules for the logical model 119

4.9 Fuzzy inference systems 122

4.9.1 Rules base 123

4.9.2 Fuzzification block 124

4.9.3 Inference block 125

4.9.4 Defuzzification block 131

4.10 Application of fuzzy sets 134

4.10.1 Fuzzy Delphi method 134

4.10.2 Weighted fuzzy Delphi method 138

4.10.3 Fuzzy PERT method 139

4.10.4 Decision making in a fuzzy environment 142

4.11 Notes 153

5 Methods of knowledge representation using type-2 fuzzy sets155

5.1 Introduction 155

5.2 Basic definitions 156

5.3 Footprint of uncertainty 160

5.4 Embedded fuzzy sets 162

5.5 Basic operations on type-2 fuzzy sets 164

5.6 Type-2 fuzzy relations 169

5.7 Type reduction 172

5.8 Type-2 fuzzy inference systems 178

5.8.1 Fuzzification block 178

5.8.2 Rules base 180

5.8.3 Inference block 180

5.9 Notes 186

6 Neural networks and their learning algorithms 187

6.1 Introduction 187

6.2 Neuron and its models 188

6.2.1 Structure and functioning of a single neuron 188

6.2.2 Perceptron 190

6.2.3 Adaline model 196

6.2.4 Sigmoidal neuron model 202

6.2.5 Hebb neuron model 206

6.3 Multilayer feed-forward networks 208

6.3.1 Structure and functioning of the network 208

6.3.2 Backpropagation algorithm 210

6.3.3 Backpropagation algorithm with momentum term 218

6.3.4 Variable-metric algorithm 220

6.3.5 Levenberg-Marquardt algorithm 221

6.3.6 Recursive least squares method 222

6.3.7 Selection of network architecture 225

6.4 Recurrent neural networks 232

6.4.1 Hopfield neural network 232

6.4.2 Hamming neural network 236

6.4.3 Multilayer neural networks with feedback 238

6.4.4 BAM network 238

6.5 Self-organizing neural networks with competitive learning 240

6.5.1 WTA neural networks 240

6.5.2 WTM neural networks 246

6.6 ART neural networks 250

6.7 Radial-basis function networks 254

6.8 Probabilistic neural networks 261

6.9 Notes 263

7 Evolutionary algorithms 265

7.1 Introduction 265

7.2 Optimization problems and evolutionary algorithms 266

7.3 Type of algorithms classified as evolutionary algorithms 267

7.3.1 Classical genetic algorithm 268

7.3.2 Evolution strategies 289

7.3.3 Evolutionary programming 307

7.3.4 Genetic programming 309

7.4 Advanced techniques in evolutionary algorithms 310

7.4.1 Exploration and exploitation 310

7.4.2 Selection methods 311

7.4.3 Scaling the fitness function 314

7.4.4 Specific reproduction procedures 315

7.4.5 Coding methods 317

7.4.6 Types of crossover 320

7.4.7 Types of mutation 322

7.4.8 Inversion 323

7.5 Evolutionary algorithms in the designing of neural networks 323

7.5.1 Evolutionary algorithms applied to the learning of weights of neural networks 324

7.5.2 Evolutionary algorithms for determining the topology of the neural network 327

7.5.3 Evolutionary algorithms for learning weights and determining the topology of the neural network 330

7.6 Evolutionary algorithms vs fuzzy systems 332

7.6.1 Fuzzy systems for evolution control 333

7.6.2 Evolution of fuzzy systems 335

7.7 Notes 344

8 Data clustering methods 349

8.1 Introduction 349

8.2 Hard and fuzzy partitions 350

8.3 Distance measures 354

8.4 HCM algorithm 357

8.5 FCM algorithm 359

8.6 PCM algorithm 360

8.7 Gustafson-Kessel algorithm 361

8.8 FMLE algorithm 363

8.9 Clustering validity measures 364

8.10 Illustration of operation of data clustering algorithms 367

8.11 Notes 369

9 Neuro-fuzzy systems of Mamdani, logical and Takagi-Sugeno type 371

9.1 Introduction 371

9.2 Description of simulation problems used 372

9.2.1 Polymerization 372

9.2.2 Modeling a static non-linear function 373

9.2.3 Modeling a non-linear dynamic object (Nonlinear Dynamic Problem - NDP) 373

9.2.4 Modeling the taste of rice 374

9.2.5 Distinguishing of the brand of wine 374

9.2.6 Classification of iris flower 374

9.3 Neuro-fuzzy systems of Mamdani type 375

9.3.1 A-type systems 375

9.3.2 B-type systems 377

9.3.3 Mamdani type systems in modeling problems 378

9.4 Neuro-fuzzy systems of logical type 390

9.4.1 M1-type systems 392

9.4.2 M2-type systems 399

9.4.3 M3-type systems 405

9.5 Neuro-fuzzy systems of Takagi-Sugeno type 410

9.5.1 M1-type systems 413

9.5.2 M2-type systems 414

9.5.3 M3-type systems 416

9.6 Learning algorithms of neuro-fuzzy systems 418

9.7 Comparison of neuro-fuzzy systems 435

9.7.1 Models evaluation criteria taking into account their complexity 437

9.7.2 Criteria isolines method 439

9.8 Notes 448

10 Flexible neuro-fuzzy systems 449

10.1 Introduction 449

10.2 Soft triangular norms 449

10.3 Parameterized triangular norms 452

10.4 Adjustable triangular norms 456

10.5 Flexible systems 461

10.6 Learning algorithms 463

10.6.1 Basic operators 470

10.6.2 Membership functions 471

10.6.3 Constraints 473

10.6.4 H-functions 473

10.7 Simulation examples 479

10.7.1 Polymerization 480

10.7.2 Modeling the taste of rice 480

10.7.3 Classification of iris flower 482

10.7.4 Classification of wine 484

10.8 Notes 492

References 495