Regression Models

Instructor: Prof. Victor Panaretos

Assistant: Dr. Kjell Konis

ANNOUNCEMENT

The midterm exam was on Wednesday, November 2nd, 16.15-18.00 in room MA30. I hope you all had a good time.

Description

Regression modelling is a basic tool of statistics, because it describes how one variable may depend on another. The aim of this course is to familiarise students with the basics of regression modelling, and some related topics. 

Topics covered include:

• Properties of the Multivariate Gaussian distribution and related quadratic forms.
• Gaussian linear regression: likelihood, least squares, variable manipulation and transformation, interactions.
• Geometrical interpretation, weighted least squares; distribution theory, Gauss-Markov theorem.
• Analysis of variance: F-statistics; sums of squares; orthogonality; experimental design.
• Linear statistical inference: general linear tests and confidence regions, simultaneous inference
• Model checking and validation: residual diagnostics, outliers and leverage points.
• Model selection: the bias variance effect, stepwise procedures. Information-based criteria.
• Multicollinearity and penalised estimation: ridge regression, the LASSO, relation to model selection, bias and variance revisited.
• Departures from standard assumptions: non-linear least Gaussian regression, robust regression and M-estimation.
• Nonparametric regression: kernel smoothing, roughness penalties, effective degrees of freedom, projection pursuit and additive models.

For references and the grading scheme click here.

Required prior knowledge

The second-year course in statistics; first-year course in linear algebra

Useful Documents

• Structuring a report (V.M. Panaretos)
• Advice on writing a report (A.C. Davison)
• Advice on writing a report (D.R. Brillinger)
• Writing technical papers or reports (A.S.C. Ehrenberg)
• Some useful notes on matrix calculus (from C.A. Felippa's online book)

Exam Information

There will be a written midterm exam and a written final exam.
No notes, books or any other material will be allowed in the exam.

Midterm

The midterm exam will be on Wednesday, November 2nd, 16.15-18.00 in MA30.

Exercises/Solutions

Each week all the theory exercises should be attempted and handed in the following week. One of these exercises will be graded S (satisfactory) or N (not satisfactory).

Practicals

Practicals aim at developing the practical skills required when applying regression methodology. Practicals are assigned on certain weeks and are due two weeks later.