Optimisation
aulus@gsu.edu.tr
Content
Objective
Our first aim in this course is to learn the mathematical construction and solution methods of optimization problems under constraints or without constraints. Secondly, it is to address the optimization problems encountered in Data Science as an application.
Course Content
Introduction to Mathematical Definitions and Concepts
Convexity
Derivative
Taylor polynomials
Unconstrained Optimization
Local vs global problem
Primary and secondary conditions
Algorithms, two basic strategies: line search and trust region
Least Squares Problems-Regression Application
Optimization Under Constraints
feasible region
Equality constraint-Inequality constraint and Lagrange method
Geometric View
Linear programming-Quadratic Programming
Simplex method, dual problem
Interior points method
Application: Machine Learning Problems
Clustering-Binary classification-Audio processing-Recommendation Systems-Logistic correlation-Deep learning-Artificial neural networks..etc.
Course Learning Outcomes
In order to understand and develop Data Science problems, it is necessary to assimilate and learn the knowledge and methodologies of fields such as Mathematics, Statistics, and Programming. Optimization itself is a very broad, very old field of study in both mathematics and engineering. A significant portion of optimization problems are used to understand Data Science problems in the following two ways.
1) Applying optimization methods to Data Science problems:
We use optimization methods when necessary in Data Science problems (somewhere, the shortest path, the least costly job, predictions, etc.). Example: When training Artificial Neural networks.
2) Using data as input for optimization problems:
We try to adapt the model to the data so that model simulations can give real results. Example: Regression
Student who passed this course
1) Remembers the basics of optimization.
2) Learns to classify optimization problems, for example linear/non-linear; discrete/continuous, convex/non-convex, unconstrained/under constraints.. etc. Learns the methods in the content.
3) Learns which method works appropriately for which Data Science problem.
Teaching and Learning Methods
Course, Problem Solving and Programming Applications
References
Numerical Optimization, J. Nocedal& S. J. Wright, Springer, 1999. ve 2. basım: 2006.
Introduction to Global Optimization, R. Horst , P. M.Pardolas&N. V. Thoai, Kluwer Academic Publishers, 1995.
The Princeton Companion to Applied Mathematics, Edited by Nicholas J. Higham, Princeton University Press, 2015
https://nhigham.com/2016/03/29/the-top-10-algorithms-in-applied-mathematics/
Linear Programming and Network Flows, Mokhtar S. Bazaraa, John J. Jarvis, Hanif D. Sherali. John Wiley, 2004. Third edition
A gentle introduction to optimization / B. Guenin , J. Könemann , L. Tunçel Cambridge
University Press
http://www.veridefteri.com/: en güncel kaynaklar, ders notları, haber, bilimsel programlama
Weekly Contents
| Week | Weekly Contents |
|---|---|
| 1 | Introduction to the course syllabus and the relationship between Data Science and Optimization |
| 2 | Introduction to Mathematical Definitions and Concepts, Convexity. Derivative. Taylor polynomials. |
| 3 | Unrestricted Optimization. Local vs global problem. Primary and secondary conditions. Problem Application. |
| 4 | Numerical Methods and Algorithms. Least Squares Problems-Regression Application. |
| 5 | Optimization Under Constraints, Feasible region, Equality-Inequality constraints. Lagrange multiplier method. |
| 6 | Geometric View and Applications |
| 7 | Midterm |
| 8 | Linear programming. Simplex method, dual problem |
| 9 | Quadratic Programming. Problems. |
| 10 | Application: Artificial Learning Problems Clustering-Binary classification-Audio processing-Recommendation Systems-Logistic correlation-Deep learning-Artificial neural networks..etc |
| 11 | Application: Artificial Learning Problems Clustering-Binary classification-Audio processing-Recommendation Systems-Logistic correlation-Deep learning-Artificial neural networks..etc |
| Week | Weekly Contents |
|---|---|
| 1 | First Examples which reveals the relationship between Data Science and Optimization |
| 2 | Applications, problems and student presentations of the theoretical course for the relevant week |
| 3 | Applications, problems and student presentations of the theoretical course for the relevant week |
| 4 | Applications, problems and student presentations of the theoretical course for the relevant week |
| 5 | Applications, problems and student presentations of the theoretical course for the relevant week |
| 6 | Applications, problems and student presentations of the theoretical course for the relevant week |
| 7 | Midterm |
| 8 | Applications, problems and student presentations of the theoretical course for the relevant week |
| 9 | Applications, problems and student presentations of the theoretical course for the relevant week |
| 10 | Applications, problems and student presentations of the theoretical course for the relevant week |
| 11 | Applications, problems and student presentations of the theoretical course for the relevant week |
Assessment System
| Activities | Number | Contribution |
|---|---|---|
| Contribution of in-term studies to overall grade | 3 | 60 |
| Contribution of final exam to overall grade | 1 | 40 |
| Total | 4 | 100 |
| Activities | Number | Contribution |
|---|---|---|
| Assignments | 1 | 15 |
| Presentation | 1 | 15 |
| Midterm Examinations (including preparation) | 1 | 30 |
| Project | 0 | 0 |
| Laboratory | 0 | 0 |
| Other Applications | 0 | 0 |
| Quiz | 0 | 0 |
| Term Paper/ Project | 0 | 0 |
| Portfolio Study | 0 | 0 |
| Reports | 0 | 0 |
| Learning Diary | 0 | 0 |
| Thesis/ Project | 0 | 0 |
| Seminar | 0 | 0 |
| Other | 0 | 0 |
| Make-up | 0 | 0 |
| Total | 3 | 60 |
ECTS
| Activities | Number | Period | Total Workload |
|---|---|---|---|
| Class Hours | 12 | 4 | 48 |
| Working Hours out of Class | 12 | 6 | 72 |
| Assignments | 2 | 8 | 16 |
| Presentation | 1 | 5 | 5 |
| Midterm Examinations (including preparation) | 4 | 12 | 48 |
| Project | 0 | 0 | 0 |
| Laboratory | 0 | 0 | 0 |
| Other Applications | 0 | 0 | 0 |
| Final Examinations (including preparation) | 0 | 0 | 0 |
| Quiz | 0 | 0 | 0 |
| Term Paper/ Project | 0 | 0 | 0 |
| Portfolio Study | 0 | 0 | 0 |
| Reports | 0 | 0 | 0 |
| Learning Diary | 0 | 0 | 0 |
| Thesis/ Project | 0 | 0 | 0 |
| Seminar | 0 | 0 | 0 |
| Other | 0 | 0 | 0 |
| Make-up | 0 | 0 | 0 |
| Yıl Sonu | 0 | 0 | 0 |
| Hazırlık Yıl Sonu | 0 | 0 | 0 |
| Hazırlık Bütünleme | 0 | 0 | 0 |
| Total Workload | 189 | ||
| Total Workload / 25 | 7.56 | ||
| Credits ECTS | 8 | ||