Advanced Statistical Modeling (ISI 514) | GSÜ

Content

Objective

The aim of this course is to enable students to learn and apply advanced methods in the field of statistical modeling. Students will delve deeper into the concepts of probability and sampling, learn the generation of random variables, exploratory data analysis, and use Monte Carlo methods for inferential statistics. In addition, they will gain extensive knowledge and skills on data partitioning, probability density estimation, supervised and unsupervised learning techniques, and parametric and nonparametric models.

Course Content

Probability Concepts, Sampling Concepts, Generating Random Variables, Exploratory Data Analysis, Finding Structure, Monte Carlo Methods for Inferential Statistics, Data Partitioning, Probability Density Estimation, Supervised Learning, Unsupervised Learning, Parametric and Nonparametric Models.

Course Learning Outcomes

Upon successful completion of this course, a student will be able to:
LO 1: Understand fundamental probability concepts, including random variables, probability distributions, and conditional probability; Apply probability theory to real-world scenarios, such as risk assessment and decision-making.
LO 2: Master advanced statistical inference methods, including maximum likelihood estimation, hypothesis testing, and confidence intervals; Evaluate the performance of different inference techniques and choose appropriate methods for specific problems.
LO 3: Explore and visualize complex datasets using techniques like scatter plots, histograms, and box plots; Identify patterns, outliers, and relationships within data.
LO 4: Gain proficiency in supervised and unsupervised learning algorithms (e.g., regression, classification, clustering); Build predictive models using techniques like decision trees and support vector machines.
LO 5: Evaluate model performance using metrics; understand overfitting, bias-variance trade-off, and regularization techniques.
LO 6: Utilize statistical software (e.g., Matlab) to implement and analyze statistical methods; Interpret and communicate results effectively to stakeholders.

References

• M.H. DeGroot and M.J. Schervish, “Probability and Statistics”, Pearson, 4th Edition, 2012.
• D.S. Moore, G.P. McCabe and B.A. Craig, “Introduction to the Practice of Statistics”, MacMillan, 10th Edition, 2021.
• S.M. Ross, “Simulation”, Academic Press, 6th Edition, 2023.
• W.L. Martinez, A.R. Martinez and J. Solka, “Exploratory Data Analysis with MATLAB”, Taylor & Francis, 2017.
• T. Hastie, R. Tibshirani and J. Friedman, “The Elements of Statistical Learning: Data Mining, Inference, and Prediction”, Springer, Second Edition, 2009.
• P. Glasserman, “Monte Carlo Methods in Financial Engineering”, Springer, 2003.
• B. Efron and R.J. Tibshirani, “An Introduction to the Bootstrap”, Chapman & Hall, 1993.
• C.M. Bishop, “Pattern Recognition and Machine Learning”, Springer, 2006.
• R.O. Duda, P.E. Hart and D.G. Stork, “Pattern Classification”, Wiley, 2nd Edition, 2001.
• J. Han, M. Kamber and J. Pei, “Data Mining: Concepts and Techniques”, Morgan Kaufmann, 3rd Edition, 2011.
• N.R. Draper and H. Smith, “Applied Regression Analysis”, Wiley-Interscience, 3rd edition, 1998.

Weekly Contents

Theory Topics
Week	Weekly Contents
1	Probability Concepts (Probability, Conditional Probability and Independence, Expectation, Common Distributions)
2	Sampling Concepts (Sampling Terminology and Concepts, Sampling Distributions, Parameter Estimation, Empirical Distribution Function)
3	Generating Random Variables (General Techniques for Generating Random Variables, Generating Continuous and Discrete Random Variables)
4	Exploratory Data Analysis (Exploring Univariate, Bivariate, Trivariate and Multi-Dimensional Data)
5	Finding Structure (Projecting Data, Principal Component Analysis, Projection Pursuit EDA, Independent Component Analysis, Nonlinear Dimensionality Reduction)
6	Monte Carlo Methods for Inferential Statistics (Classical Inferential Statistics, Monte Carlo Methods for Inferential Statistics, Bootstrap Methods)
7	Data Partitioning (Cross-Validation, Jackknife, Better Bootstrap Confidence Intervals, Jackknife-After-Bootstrap)
8	Probability Density Estimation (Histograms, Kernel Density Estimation, Finite Mixtures)
9	Midterm Exam
10	Supervised Learning (Bayes Decision Theory, Evaluating the Classifier, Classification Trees, Combining Classifiers, Nearest Neighbor Classifier, Support Vector Machines)
11	Unsupervised Learning (Measures of Distance, Hierarchical Clustering, K-Means Clustering, Model-Based Clustering, Assessing Cluster Results)
12	Parametric and Nonparametric Models (Spline Regression Models, Logistic Regression, Generalized Linear Models, Model Selection and Regularization)
13	Parametric and Nonparametric Models (Partial Least Squares Regression, Some Smoothing Methods, Kernel Methods, Smoothing Splines)
14	Parametric and Nonparametric Models (Nonparametric Regression, Regression Trees, Additive Models, Multivariate Adaptive Regression Splines)

Assessment System

Contribution to Overall Grade
Activities	Number	Contribution
Contribution of in-term studies to overall grade	8	55
Contribution of final exam to overall grade	1	45
Total	9	100

In-Term Studies
Activities	Number	Contribution
Assignments	7	35
Presentation	0	0
Midterm Examinations (including preparation)	1	20
Project	0	0
Laboratory	0	0
Other Applications	0	0
Quiz	0	0
Term Paper/ Project	0	0
Portfolio Study	0	0
Reports	0	0
Learning Diary	0	0
Thesis/ Project	0	0
Seminar	0	0
Other	0	0
Make-up	0	0
Total	8	55

Relation of Proficiency

No	Program Learning Outcomes	Contribution
No	Program Learning Outcomes	1	2	3	4	5
1					X
2						X
3					X
4					X
5				X
6
7						X
8
9				X
10
11
12				X

ECTS

Activities	Number	Period	Total Workload
Class Hours	14	3	42
Working Hours out of Class	13	2	26
Assignments	7	5	35
Presentation	0	0	0
Midterm Examinations (including preparation)	1	10	10
Project	0	0	0
Laboratory	0	0	0
Other Applications	0	0	0
Final Examinations (including preparation)	1	35	35
Quiz	0	0	0
Term Paper/ Project	0	0	0
Portfolio Study	0	0	0
Reports	0	0	0
Learning Diary	0	0	0
Thesis/ Project	0	0	0
Seminar	0	0	0
Other	0	0	0
Make-up	0	0	0
Yıl Sonu	0	0	0
Hazırlık Yıl Sonu	0	0	0
Hazırlık Bütünleme	0	0	0
Total Workload			148
Total Workload / 25			5.92
Credits ECTS			6