Skip to
content

main University navigation

local section navigation

search
Study
Research
Business
International
Birmingham
Alumni
About

Open Days

Aston Online
More
Study
Course finder
Placement year
Languages for all
Scholarships
Open days
Prospectus
Undergraduate
Postgraduate
Teachers
Aston Online
Research
REF results
Research degrees
Aston Research Explorer
Graduate School
Research in use
Business
Short courses
Employee development
Hire a placement student
Conference facilities
Aston Centre for Growth
Enterprise Research Centre
Goldman Sachs 10,000 Small Businesses
International
Students
Exchanges
Partners & development
Agents
Aston in your country
Scholarships
English language programmes
Muscat University
Birmingham
Student life & satisfaction
Student bloggers
City living
Visiting
Alumni
Aston 50
Alumni in your country
Benefits & Services
Get Involved
Make a Gift
Stay in contact
About
Social media
News
Events
League tables
History
Map & directions
Term dates
Accommodation
Current students
Prospective students
International guarantee
Types of accommodation
Apply for accommodation
Careers+Placements
Aston Futures
Placement support
Graduate support
CV guidance
Employer services
Get advice & contact us
Student Resources
My Aston Portal
Blackboard VLE
Student Email
myAston App
Pay Outstanding Invoices
Make Payment Before Arrival
Online Store
Engineering & Applied Science
Research
Research groups
Nonlinearity and Complexity Research Group
Resources
Netlab
Book
.
Netlab: Algorithms for Pattern Recognition
Chapter 1: Introduction
Introduction to Matlab: Matlab Basics; Matlab Plotting; Programming in Matlab; Programming Facilities.
The Netlab Toolbox: Overview of Netlab; Generic Functions.
Worked Example: Data Normalisation
Chapter 2: Parameter Optimisation Algorithms
Controlling the Algorithms: Information Display; Termination Criteria; Extra Arguments and Return Values; Demonstration Program. Quadratic Approximation at a Minimum Line Search: Precision; Line Minimisation Algorithms; Bracketing the Minimum. Batch Gradient Descent Conjugate Gradients Scaled Conjugate Gradients QuasiNewton Methods Optimisation and Neural Networks: General Purpose Optimisation Algorithms; Online Gradient Descent. Worked Example: Constrained Optimisation
Chapter 3: Density Modelling and Clustering
Gaussian Mixture Models: Theory; Implementation.
Computing Probabilities: GMM Activations; GMM Probabilities; GMM Posteriors.
EM Training Algorithm: Algorithm Theory; Initialisation; Netlab EM Implementation.
Demonstrations of GMM: EM Algorithm; Density Modelling.
Kmeans Clustering: Algorithm and Netlab Implementation; Demonstration Program.
Knearestneighbour: Algorithm and Netlab Implementation; Demonstration Program.
Worked Examples: Classification with Density Models; Novelty Detection.
Chapter 4: Single Layer Networks
The Single Layer Feedforward Network: Netlab Implementation; Forward Propagation. Error Functions Error Gradient Calculation Evaluating Other Derivatives: Network Activation Derivatives; The Hessian Matrix. Iterated Reweighted Least Squares Training Demonstration Programs: Twoclass Problem; Threeclass Problem. Worked Example: Training Regularised Models
Chapter 5: The Multilayer Perceptron
The Twolayer Feedforward Network: Definition; Network Creation and Initialisation; Manipulating Weights; Forward Propagation. Error Functions and Network Training Error Gradient Calculation Evaluating other Derivatives The Hessian Matrix: Fast Multiplication by the Hessian; Netlab Implementation. Demonstration Programs: Regression Demonstration; Classification Demonstration. Mixture Density Networks: Model Structure; Network Creation and Initialisation; Forward Propagation; Conditional Probabilities; Error and Gradient Calculation; Demonstration Program. Worked Example: Adding Direct Connections
Chapter 6: Radial Basis Functions
The RBF Network: Theory; Netlab}\ Implementation. Special Purpose Training Algorithms: Basis Function Optimisation; Output Weight Optimisation; Netlab Implementation. Error and Error Gradient Calculation Evaluating Other Derivatives: Output Derivatives; Network Jacobian; Network Hessian. Demonstration Program Worked Examples: Linear Smoothing; Dual Basis Functions; Characterising Network Complexity.
Chapter 7: Visualisation and Latent Variable Models
Principal Component Analysis: Theory; Netlab Implementation; Examples. Probabilistic Principal Component Analysis: Probabilistic PCA; PPCA Implementation; Mixture of PPCA; Mixture of PPCA Implementation. Generative Topographic Mapping: The GTM Model; Model Creation and Initialisation; Computing Probabilities; EM Algorithm; Magnification Factors; Demonstration Programs. Topographic Projection: Neuroscale Algorithm; Neuroscale Implementation; Demonstration of Neuroscale. Worked Example: Canonical Variates.
Chapter 8: Sampling
Monte Carlo Integration Basic Sampling: Random Number Generators; Transformation Methods; Rejection Sampling; Importance Sampling. Markov Chain Sampling: Markov Chain Fundamentals; Gibbs Sampling; MetropolisHastings Algorithm; Hybrid Monte Carlo. Demonstration Programs: MetropolisHastings Sampling; Hybrid Monte Carlo Sampling. Worked Example: Convergence Diagnostics
Chapter 9: Bayesian Techniques
Principles of Bayesian Inference Priors for Neural Networks: Theory of Priors; Netlab Implementation of Priors; Demonstration of Gaussian Priors; Masks and Weight Manipulation. Computing Error and Gradient Functions The Evidence Procedure: Theory; Netlab Implementation. Predictions and Error Bars: Predictions for Regression; Predictions for Classification; Netlab Implementation. Demonstrations of Evidence Procedure: The Evidence Procedure for Regression; The Evidence Procedure for Classification; Automatic Relevance Determination. Monte Carlo Methods Demonstration of Hybrid Monte Carlo for MLPs Worked Example: Improved Classification Approximation
Chapter 10: Gaussian Processes
Bayesian Regression: Weight Space View; Function Space View. Theory of Gaussian Processes: Model Definition; Learning Hyperparameters. Netlab Implementation: Model Creation and Initialisation; Making Predictions; Model Training. Demonstration Programs: Regression Demonstration; ARD Demonstration. Worked Example: GPs for Classification
Appendices: Linear Algebra and Matrices; Algorithm Error Analysis.
Engineering & Applied Science
Research
Research groups
Nonlinearity and Complexity Research Group
Resources
Netlab
Book
Contents
Worked Examples