Optimal Control Theory

ESE-680 Fall 2011

Welcome to the website of ESE-680 Fall 2011

Optimal Control Theory

Time and Location

Mondays and Wednesdays, 1:30-3:00 PM, in Towne 307.


Prof. George J. Pappas, Department of Electrical and Systems Engineering.


Calculus of Variations and Optimal Control Theory: A Concise Introduction, by Daniel Liberzon.

Other books/papers will be provided.

Course Contents

The goal of this special topics course is to read the textbook. Topics will include:

  1. Introduction
    The goals of the course; path optimization vs. point optimization; basic facts from finite-dimensional optimization.

  2. Calculus of variations
    Examples of variational problems; Euler-Lagrange equation; Hamiltonial formalism and Legendre transformation; mechanical interpretation; constraints; second variation and Legendre's necessary condition; weak and strong extrema; conjugate points and sufficient conditions.

  3. The maximum principle
    Statement of the optimal control problem; variational argument and preview of the maximum principle; statement and proof of the maximum principle; relation to Lie brackets; bang-bang and singular optimal controls.

  4. Hamilton-Jacobi-Bellman equation Dynamic programming; sufficient conditions for optimality; viscosity solutions of the HJB equation.

  5. LQR problems

  6. Other topics
    Optimal control on manifolds, model-predictive control, differential game theory, stochastic control.


ESE 500 Linear Systems is required. ESE 617 Nonlinear systems is suggested but not required. Please skim through the book to determine whether you feel comfortable with the text.


This class will be reading style format where research-oriented students will lead the presentation of each lecture. Depending on the number of students, there may be a class project and/or a take home final exam.