Stochastic Processes(IND 621)
| Course Code | Course Name | Semester | Theory | Practice | Lab | Credit | ECTS |
|---|---|---|---|---|---|---|---|
| IND 621 | Stochastic Processes | 1 | 3 | 0 | 0 | 3 | 8 |
| Prerequisites | |
| Admission Requirements |
| Language of Instruction | English |
| Course Type | Compulsory |
| Course Level | Doctoral Degree |
| Course Instructor(s) | EBRU ANGÜN ebru.angun@gmail.com (Email) |
| Assistant | |
| Objective |
Uncertainty exists in most real life problems, and for a better decision making, it is important to take into account the uncertainties. The use of stochastic variables is a common way to represent the uncertainty in quantities of interest such as customer demands, production lead times, product price, repair times, service times etc. that can be encountered in the decision problems in both service sector and industry. The primary aim of this course is to teach how to formulate and solve the stochastic decision problems using stochastic processes. The objectives of this course can be listed as follows. 1. To make the students be aware of the stochastic nature of most real life problems. 2. To provide students an insight of how to analyze the stochastic systems. 3. To make the students develop the necessary skills to identify, formulate and solve the stochastic problems. |
| Content |
Week 1. Review on probability concepts (Ross, Chapter 1) Week 2. Random variables: discrete and continuous, expectation, variance (Ross, Chapter 2) Week 3. Random variables (cont.): Jointly distributed random variables, variance and covariance of sum of random variables, moment generating functions, limit theorems (Ross, Chapter 2) Week 4. Conditional Probability, Conditional Expectation : Conditional distribution functions, Use of conditioning for calculating probability, expectation and variance (Ross, Chapter 3) Week 5. Markov Chain: Definition of Markov chain, Chapman-Kolmogorov Equations, Calculation of state probabilities (Ross, Chapter 4) Week 6. Markov Chain (Cont.): Classification of states, Limiting state probabilities (Ross, Chapter 4) Week 7. Discrete-Time Markov Process: State probability calculations, Numerical examples (Howard, Chapters 1-2) Week 8. Discrete-Time Markov Process with Reward (Howard, Chapter 3) Week 9. Midterm Exam Week 10. Discrete-Time Markov Decision Process : Solution algorithms: Value iteration technique for finite horizon, policy iteration algorithm for infinite horizon (Howard, Chapter 4) Week 11. Exponential Distribution : definition, properties and use of exponential distribution (Ross, Chapter 5) Week 12. Poisson Process: definition, properties of Poisson Processes, Non-homogenous and Compound Poisson Processes (Ross, Chapter 5) Week 13. Continuous-Time Markov Chain (Ross, Chapter 6) Week 14. Project presentations on the application of MDP to research problems |
| Course Learning Outcomes |
ÖÇ 1: be able to define the basic probability concepts. ÖÇ 2: be able to use the conditioning to calculate the probability of a random event or the expectation and variance of stochastic variables. ÖÇ 3: be able to identify the stochastic problems. ÖÇ 4: be able to classify the stochastic problems into appropriate categories such as Poisson or Markov Processes. ÖÇ 5: be able to formulate and solve the stochastic problems as Poisson Processes. ÖÇ 6: be able to analyze Markov chains. ÖÇ 7: be able to formulate the stochastic decision problems as discrete-time Markov decision processes (MDP) ÖÇ 8: be able to use an appropriate algorithm to solve the finite and infinite horizon MDP problems ÖÇ 9: be able to formulate and solve continuous-time Markov Processes. ÖÇ 10: be able to write a report on the analysis of a stochastic problem and do an oral presentation. |
| Teaching and Learning Methods | |
| References |
1. Ross, S., “Introduction to Probability Models”, 9th edition, Academic Press, Inc.,2007. 2. Howard, R.A., “Dynamic Programming and Markov Processes”, MIT Press, 1960. 3. Winston, W.L., “Introduction to Probability Models - Operations Research: Volume 2”, Duxbury Resource Center, 2003. |
Theory Topics
| Week | Weekly Contents |
|---|---|
| 1 | Review on probability concepts (Ross, Chapter 1) |
| 2 | Random variables: discrete and continuous, expectation, variance (Ross, Chapter 2) |
| 3 | Random variables (cont.): Jointly distributed random variables, variance and covariance of sum of random variables, moment generating functions, limit theorems (Ross, Chapter 2) |
| 4 | Conditional Probability, Conditional Expectation : Conditional distribution functions, Use of conditioning for calculating probability, expectation and variance (Ross, Chapter 3) |
| 5 | Markov Chain: Definition of Markov chain, Chapman-Kolmogorov Equations, Calculation of state probabilities (Ross, Chapter 4) |
| 6 | Markov Chain (Cont.): Classification of states, Limiting state probabilities (Ross, Chapter 4) |
| 7 | Discrete-Time Markov Process: State probability calculations, Numerical examples (Howard, Chapters 1-2) |
| 8 | Discrete-Time Markov Process with Reward (Howard, Chapter 3) |
| 9 | Midterm Exam |
| 10 | Discrete-Time Markov Decision Process : Solution algorithms: Value iteration technique for finite horizon, policy iteration algorithm for infinite horizon (Howard, Chapter 4) |
| 11 | Exponential Distribution : definition, properties and use of exponential distribution (Ross, Chapter 5) |
| 12 | Poisson Process: definition, properties of Poisson Processes, Non-homogenous and Compound Poisson Processes (Ross, Chapter 5) |
| 13 | Continuous-Time Markov Chain (Ross, Chapter 6) |
| 14 | Project presentations on the application of MDP to research problems |
Practice Topics
| Week | Weekly Contents |
|---|
Contribution to Overall Grade
| Number | Contribution | |
|---|---|---|
| Contribution of in-term studies to overall grade | 0 | 60 |
| Contribution of final exam to overall grade | 0 | 40 |
| Toplam | 0 | 100 |
In-Term Studies
| Number | Contribution | |
|---|---|---|
| Assignments | 4 | 20 |
| Presentation | 0 | 0 |
| Midterm Examinations (including preparation) | 1 | 30 |
| Project | 1 | 10 |
| Laboratory | 0 | 0 |
| Other Applications | 0 | 0 |
| Quiz | 0 | 0 |
| Term Paper/ Project | 0 | 0 |
| Portfolio Study | 0 | 0 |
| Reports | 0 | 0 |
| Learning Diary | 0 | 0 |
| Thesis/ Project | 0 | 0 |
| Seminar | 0 | 0 |
| Other | 0 | 0 |
| Toplam | 6 | 60 |
| No | Program Learning Outcomes | Contribution | ||||
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | ||
| 1 | X | |||||
| 2 | X | |||||
| 3 | X | |||||
| 4 | X | |||||
| 5 | X | |||||
| 6 | ||||||
| 7 | ||||||
| 8 | X | |||||
| 9 | ||||||
| 10 | ||||||
| Activities | Number | Period | Total Workload |
|---|---|---|---|
| Class Hours | 14 | 3 | 42 |
| Working Hours out of Class | 13 | 5 | 65 |
| Assignments | 4 | 10 | 40 |
| Midterm Examinations (including preparation) | 1 | 10 | 10 |
| Project | 1 | 40 | 40 |
| Final Examinations (including preparation) | 1 | 17 | 17 |
| Total Workload | 214 | ||
| Total Workload / 25 | 8,56 | ||
| Credits ECTS | 9 | ||