Plausible Neural Networks for Biological Modelling Book

	Preface	1
Part I	Fundamentals
1	Biological Evidence for Synapse Modification Relevant for Neural Network Modelling
1.	Introduction	7
2.	The Synapse	11
3.	Long Term Potentiation	13
4.	Two Characteristic Types of Experiment	15
4.1	Food Discrimination Learning in Chicks	15
4.2	Electrical Stimulation of Nervous Cell Cultures	18
5.	Conclusion	19
	References and Further Reading	20
2	What is Different with Spiking Neurons?
1.	Spikes and Rates	23
1.1	Temporal Average-Spike Count	24
1.2	Spatial Average-Population Activity	26
1.3	Pulse Coding-Correlations and Synchrony	27
2.	'Integrate and Fire' Model	28
3.	Spike Response Model	30
4.	Rapid Transients	33
5.	Perfect Synchrony	36
6.	Coincidence Detection	38
7.	Spike Time Dependent Hebbian Learning	39
8.	Temporal Coding in the Auditory System	42
9.	Conclusion	43
	References	45
3	Recurrent Neural Networks: Properties and Models
1.	Introduction	49
2.	Universality of Recurrent Networks	52
2.1	Discrete Time Dynamics	52
2.2	Continuous Time Dynamics	54
3.	Recurrent Learning Algorithms for Static Tasks	56
3.1	Hopfield Network	56
3.2	Boltzmann Machines	58
3.3	Recurrent Backpropagation Proposed by Fernando Pineda	60
4.	Recurrent Learning Algorithms for Dynamical Tasks	63
4.1	Backpropagation Through Time	63
4.2	Jordan and Elman Networks	64
4.3	Real Time Recurrent Learning (RTRL)	65
4.3.1	Continuous Time RTRL	65
4.3.2	Discrete Time RTRL	66
4.3.3	Teacher Forced RTRL	67
4.3.4	Considerations about the Memory Requirements	67
4.4	Time Dependent Recurrent Backpropagation (TDRBP)	68
5.	Other Recurrent Algorithms	69
6.	Conclusion	70
	References	72
4	A Derivation of the Learning Rules for Dynamic Recurrent Neural Networks
1.	A Look into the Calculus of Variations	75
2.	Conditions of Constraint	77
3.	Applications in Physics: Lagrangian and Hamiltonian Dynamics	78
4.	Generalized Coordinates	80
5.	Application to Optimal Control Systems	82
6.	Time Dependent Recurrent Backpropagation: Learning Rules	85
	References	88
Part II	Applications to Biology
5	Simulation of the Human Oculomotor Integrator Using a Dynamic Recurrent Neural Network
1.	Introduction	92
2.	The Different Neural Integrator Models	95
3.	The Biologically Plausible Improvements	99
3.1	Fixed Sign Connection Weights	100
3.2	Artificial Distance between Inter-Neurons	101
3.3	Numerical Discretization of the Continuous Time Model	101
3.4	The General Supervisor	102
3.5	The Modified Network	103
4.	Emergence of Clusters	104
4.1	Definition	105
4.2	Mathematical Identification of Clusters	106
4.3	Characterization of the Clustered Structure	106
4.4	Particular Locations	110
5.	Discussion and Conclusion	110
	References	112
6	Pattern Segmentation in an Associative Network of Spiking Neurons
1.	The Binding Problem	117
2.	Spike Response Model	118
3.	Simulation Results	121
3.1	Pattern Retrieval and Synchronization	123
3.2	Pattern Segmentation	124
3.3	Context Sensitive Binding in a Layered Network with Feedback	126
4.	Related Work	129
4.1	Segmentation with LEGION	129
4.2	How about Real Brains?	130
	References	131
7	Cortical Models for Movement Control
1.	Introduction: Constraints on Modeling Biological Neural Networks	135
2.	Cellular Firing Patterns in Monkey Cortical Areas 4 and 5	137
3.	Anatomical Links between Areas 4 and 5, Spinal Motoneurons, and Sensory Systems	140
4.	How Insertion of a Time Delay can Create a Niche for Deliberation	141
5.	A Volition-Deliberation Nexus and Voluntary Trajectory Generation	142
6.	Cortical-Subcortical Cooperation for Deliberation and Task-Dependent Configuration	146
7.	Cortical Layers, Neural Population Codes, and Posture-Dependent Recruitment of Muscle Synergies	150
8.	Trajectory Generation in Handwriting and Viapoint Movements	151
9.	Satisfying Constraints of Reaching to Intercept or Grasp	155
10.	Conclusions: Online Action Composition by Cortical Circuits	156
	References	157
8	Implications of Activity Dependent Processes in Spinal Cord Circuits for the Development of Motor Control; a Neural Network Model
1.	Introduction	164
2.	Sensorimotor Development	165
3.	Reflex Contributions to Joint Stiffness	166
4.	The Model	167
4.1	Neural Model	168
4.2	Musculo-Skeletal Model	170
4.3	Muscle Model	172
4.4	Sensory Model	173
4.5	Model Dynamics	174
5.	Experiments	174
5.1	Training	176
5.2	Neural Control Properties	177
5.3	Perturbation Experiments	179
6.	Discussion	182
	References	185
9	Cortical Maps as Topology-Representing Neural Networks Applied to Motor Control: Articulatory Speech Synthesis
1.	Lateral Connections in Cortical Maps	190
2.	A Neural Network Model	191
3.	Spatial Maps as Internal Representations for Motor Planning	193
3.1	Dynamical Behavior of Spatial Maps	194
3.2	Function Approximation by Interconnected Maps	196
3.3	Dynamical Inversion	199
4.	Application of Cortical Maps to Articulatory Speech Synthesis	200
4.1	Cortical Control of Speech Movements	202
4.2	An Experimental Study	203
4.2.1	The Training Procedure	204
4.2.2	Field Representation of Phonemic Targets	208
4.2.3	Non-Audible Gestures and Compensation	211
4.2.4	Generation of VVV ... Sequences	211
5.	Conclusions	215
	References	216
10	Line and Edge Detection by Curvature-Adaptive Neural Networks
1.	Introduction	220
2.	Biological Constraints	223
3.	Construction of the Gabor Filters	224
4.	The One-Dimensional Case	224
5.	The Two-Dimensional Case	225
6.	Simple Detection Scheme	225
7.	An Extended Detection Scheme	226
8.	Intermezzo: A Multi-Scale Approach	230
9.	Advanced Detection Scheme	231
10.	Biological Plausibility of the Adaptive Algorithm	233
11.	Conclusion and Discussion	235
	References	238
11	Path Planning and Obstacle Avoidance Using a Recurrent Neural Network
1.	Introduction	241
2.	Problem Description	242
3.	Task Descriptions	243
3.1	Representations	243
3.2	Fusing the Representations into a Neuronal Map	245
3.3	Path Planning and Heading Decision	246
4.	Results	248
5.	Conclusions	251
	References	253
	Index	255