Course Title: MAE 5930 Optimization
Course Setting: Tuesdays and Thursdays 7:30 to 8:45 AM in ENGR 106
Office hours are Tuesdays and Thursdays 8:45 to 10 AM in ENGR 419H or by appointment
Targeted Audience: The course is targeted at seniors and early graduate students of engineering, science, and math. The course is appropriate for students preparing for industry positions and students initiating research.
Why take this course and work diligently to succeed? The answer depends on your values. Optimization is a beautiful subject that has entertained great minds for centuries. Many important contributions have been made in our lifetime. There is joy sharing in their developments. For those hoping to land your dream job, optimization problems are pervasive. They occur in economics, finance, government, and all forms of engineering. Optimization is the backbone of machine learning. Demonstrating proficient knowledge in optimization increases your employment chances. For those hoping to land high-paying jobs, the energy industry pays more than $150,000 for a PhD level optimization expert just graduating. The tech and financial industries pay substantially more.
Course Outcomes: Students preparing for industry positions should be able to recognize when an optimization problem is at hand, model it, identify basic problem structure, and select an appropriate commercial tool to solve it. Students preparing for research should have these skills as well as comfort with mathematical formalism and algorithm development.
Course Description: Students will learn the mathematical theory of optimization and computational techniques for solving problems. The theoretical topics include mathematical foundations, unconstrained optimization, the Karush-Kuhn-Tucker and Fritz John conditions for constrained problems of various forms, and convexity. The computational techniques discussed include those of unconstrained, linear, convex, nonlinear, and integer programming. The emphasis of the course is to develop strong fundamentals, modeling abilities, and solution strategies for engineering problems. About one-third of the class is dedicated to solving problems, one-third to mathematical results, and one-third to methods and algorithms.
Prerequisites: The incoming student needs a solid foundation in multivariable calculus, linear algebra, differential equations, and numerical methods. The Utah State University requirements are
MATH 2210 Multivariable Calculus
MATH 2250 Linear Algebra and Differential Equations
MAE 3210 Numerical Methods
Textbooks: You are not required to purchase a textbook. There are many good books in optimization – some of which are free online. Two good references that are freely available are
An Introduction to Optimization by Chong and Zak
Convex Optimization by Boyd
The first is available for free as an ebook through the Utah State University library. This book contains more material than can be learned in a one semester introductory course, but it is a good reference for mathematical foundations, optimization theory, neural networks, algorithms, and so on. The second book by Boyd is a great book for convex optimization, applications, and numerical methods.
Two other introductory references are
A Gentle Introduction to Optimization by Guenin, Konemann, and Tuncel
Introduction to Optimization by Pedregal
The first of these is simple but mathematically rigorous and contains a nice section on complexity. It also contains descriptions of applied problems in the exercises. The second is more elementary in nature and not as comprehensive.
Three of my favorite books are below. The last one is freely available online. Their target audiences are graduate students.
Convexity and Optimization in Rn by Berkovitz
Optimal Control Theory by Berkovitz
Calculus of Variations and Optimal Control Theory by Liberzon
Student Evaluations: Student performance is evaluated in homework assignments and exams. The purpose of the homework is to practice and learn by doing. The homework can be worked individually or in a group, but the submission must represent the individual understanding. Homework topics include mathematics, problem solving, and programming (in MATLAB or Python). Two or three assignments are given per exam. Homework is 25% of the final grade.
There are three exams. The last exam is a comprehensive final exam administered during the university specified time. Each exam is 25% of the final grade.
Exam 1. 25%
Exam 2. 25%
Exam 3. 25%
Grading is done so that the grades reflect the individual’s understanding of the course concepts. Grading is not done to reflect performance relative to peers in the class.
There are many things in life more important than a grade in Optimization class, and achieving a good grade is not worth sacrificing one’s integrity. The instructor and students are expected to adhere to the Honor Pledge: “I pledge, on my honor, to conduct myself with the foremost level of academic integrity.”
Course Schedule: The following schedule is a rough outline for the course. It is subject to change throughout the semester.
Week 1: Course overview and examples
Week 2: Unconstrained optimization
Week 3: Unconstrained optimization
Week 4: Linear programming
Week 5: Linear programming and Exam 1
Week 6: Mixed integer linear programming
Week 7: Mixed integer linear programming
Week 8: Convex programming
Week 9: Convex programming
Week 10: Convex programming
Week 11: Exam 2
Week 12: Nonlinear programming
Week 13: Nonlinear programming
Week 14: Multiobjective optimization
Week 15: Multiobjective optimization
Week 16: Exam 3