Introduction to Neural Networks

Natural Sciences > Physics > Introduction to Neural Networks

Introduction to Neural Networks posted by member7_php on 2/12/2009

Add To Favorites

Courseware/Lectures

Test/Tutorials

Spring 2005

Neurons forming a network in disassociated cell culture. (Image courtesy of Seung Laboratory, MIT Department of Brain and Cognitive Sciences.)

Course Highlights

This course features a selection of downloadable lecture notes and problem sets in the assignments section.

Course Description

This course explores the organization of synaptic connectivity as the basis of neural computation and learning. Perceptrons and dynamical theories of recurrent networks including amplifiers, attractors, and hybrid computation are covered. Additional topics include backpropagation and Hebbian learning, as well as models of perception, motor control, memory, and neural development.

Technical Requirements

Special software is required to use some of the files in this course: .mat, and .m.

*Some translations represent previous versions of courses.

Syllabus

Course Philosophy

The subject will focus on basic mathematical concepts for understanding nonlinearity and feedback in neural networks, with examples drawn from both neurobiology and computer science. Most of the subject is devoted to recurrent networks, because recurrent feedback loops dominate the synaptic connectivity of the brain. There will be some discussion of statistical pattern recognition, but less than in the past, because this perspective is now covered in Machine Learning and Neural Networks. Instead the connections to dynamical systems theory will be emphasized.

Modern research in theoretical neuroscience can be divided into three categories: cellular biophysics, network dynamics, and statistical analysis of neurobiological data. This subject is about the dynamics of networks, but excludes the biophysics of single neurons, which will be taught in 9.29J, Introduction to Computational Neuroscience.

Prerequisites

Permission of the instructor
Familiarity with linear algebra, multivariate calculus, and probability theory
Knowledge of a programming language (MATLAB® recommended)

Course Requirements

Problem sets
Midterm exam
Final exam

Textbook

The following text is recommended:

Hertz, John, Anders Krogh, and Richard G. Palmer. Introduction to the Theory of Neural Computation. Redwood City, CA: Addison-Wesley Pub. Co., 1991. ISBN: 9780201515602.

Calendar

Calendar schedule.
Lec #	Topics	Key DATES
1	From Spikes to Rates
2	Perceptrons: Simple and Multilayer
3	Perceptrons as Models of Vision
4	Linear Networks	Problem set 1 due
5	Retina
6	Lateral Inhibition and Feature Selectivity	Problem set 2 due
7	Objectives and Optimization	Problem set 3 due
8	Hybrid Analog-Digital Computation Ring Network
9	Constraint Satisfaction Stereopsis	Problem set 4 due
10	Bidirectional Perception
11	Signal Reconstruction	Problem set 5 due
12	Hamiltonian Dynamics
	Midterm
13	Antisymmetric Networks
14	Excitatory-Inhibitory Networks Learning
15	Associative Memory
16	Models of Delay Activity Integrators	Problem set 6 due one day after Lec #16
17	Multistability Clustering
18	VQ PCA	Problem set 7 due
19	More PCA Delta Rule	Problem set 8 due
20	Conditioning Backpropagation
21	More Backpropagation	Problem set 9 due
22	Stochastic Gradient Descent
23	Reinforcement Learning	Problem set 10 due
24	More Reinforcement Learning
25	Final Review
	Final Exam

Readings

Lec #	Topics	READINGS
1	From Spikes to Rates	Koch, Christof. Biophysics of Computation: Information Processing in Single Neurons. New York, NY: Oxford University Press, 2004, section 14.2, pp. 335-341. ISBN: 9780195181999. Ermentrout, Bard. "Reduction of Conductance-Based Models with Slow Synapses to Neural Nets." Neural Computation 6, no. 4 (July 1994): 679-695.
2	Perceptrons: Simple and Multilayer
3	Perceptrons as Models of Vision	Marr, David. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. New York, NY: W.H. Freeman & Company, 1983, section 2.2, pp. 54-79. ISBN: 9780716715672. Hubel, David H. Eye, Brain, and Vision. New York, NY: W.H. Freeman & Company, 1988, chapter 3, pp. 39-46. ISBN: 9780716750208. LeNet Web site
4	Linear Networks
5	Retina	Adelson, E. H. "Lightness Perception and Lightness Illusions." The New Cognitive Neurosciences. Edited by Michael S. Gazzaniga. 2nd ed. Cambridge, MA: MIT Press, 1999, pp. 339-351. ISBN: 9780262071956. Hartline, H. K., and F. Ratliff. "Inhibitory Interaction in the Retina of Limulus." Physiology of Photoreceptor Organs. Edited by Michelangelo G. F. Fuortes. New York, NY: Springer-Verlag, 1972, pp. 382-447. ISBN: 9780387057439.
6	Lateral Inhibition and Feature Selectivity	Press, William H., Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. Numerical Recipes in C: The Art of Scientific Computing. New York, NY: Cambridge University Press, 1992, chapters 12, and 13. ISBN: 9780521431088. Strang, Gilbert. Introduction to Applied Mathematics. Wellesley, MA: Wellesley-Cambridge Press, 1986, section 4.2, pp. 290-309. ISBN: 9780961408800.
7	Objectives and Optimization
8	Hybrid Analog-Digital Computation Ring Network	Hahnloser, R. H., R. Sarpeshkar, M. A. Mahowald, R. J. Douglas, and H. S. Seung. "Digital selection and analog amplification coexist in a cortex-inspired silicon circuit." Nature 405, no. 6789 (June 22, 2000): 947-51. Hahnloser, Richard H., H. Sebastian Seung, and Jean-Jacques Slotine. "Permitted and Forbidden Sets in Symmetric Threshold-Linear Networks." Neural Computation 15, no. 3 (March 2003): 621-38.
9	Constraint Satisfaction Stereopsis
10	Bidirectional Perception
11	Signal Reconstruction
12	Hamiltonian Dynamics
	Midterm
13	Antisymmetric Networks
14	Excitatory-Inhibitory Networks Learning
15	Associative Memory
16	Models of Delay Activity Integrators
17	Multistability Clustering
18	VQ PCA
19	More PCA Delta Rule
20	Conditioning Backpropagation
21	More Backpropagation
22	Stochastic Gradient Descent
23	Reinforcement Learning
24	More Reinforcement Learning
25	Final Review