Machine Learning and Deep Learning for Data Science 1
(데이터사이언스를 위한 머신러닝 딥러닝 1)
Instructor: Min-hwan Oh (minoh@snu.ac.kr, Office: 942-419) & Joonseok Lee
Goals
This course covers fundamental topics in AI and Machine learning, the“A” part of the core ABC (AI model/algorithm, Big data, Computing) courses in data science. This course covers the fundamentals of machine learning and deep learning for data scientists. Specifically, foundations of machine learning (probability, MLE, gradient descent, overfitting, regularization, etc) and basic supervised models are covered. In deep learning part, basic structure of neural networks and backpropagation will be discussed, followed by details and applications of convolutional neural networks.
Content
Part 1: Machine Learning Foundations
1. Probability, Bayes rule, Likelihood, MLE, Gradient descent (+SGD)
2. Generative classifiers (Bayes classifier)
3. Naive Bayes w/ text classification, eval metrics (Prec, recall, AUC)
4. Linear regression, feature selection
5. Overfitting, Regularization
6. Logistic regression
7. Information theory (Entropy, Mutual information, KLD), Decision trees, random forests
8. Nearest Neighbors
9. SVM
10. Boosting (+bootstrap, bagging, ensembles)
Part 2: Deep Learning Foundations
1. Softmax classifier, Cross-entropy
2. Neural networks, backpropagation
Lab 1. Handwritten digit classification, Backpropagation
3. Convolutional neural networks
4. Training neural networks I
Lab 2. Introduction to TF, Image classification, Data augmentation
5. Training neural networks II
6. Recurrent neural networks
Lab 3. Regularization, RNN
7. Seq2seq models, Attention
8. Transformers
Lab 4. Language translation, Attention, Transformers
Textbook
– ISL: “An Introduction to Statistical Learning (2nd Ed.)” by James, Witten, Hastie, and Tibshirani, 2021, Springer. (Official online copy)
– P: “Probabilistic Machine Learning: An Introduction (2nd Ed.)” by Kevin Murphy, 2021, MIT Press. (Official online copy)
– D: “Deep Learning” by Ian Goodfellow, Yoshua Bengio, Aaron Courville, 2015, MIT Press. (Official online copy)
Additional reading materials and papers will be provided.
Grading Policy
Prerequisite
– Basic Python programming: If you are not sure, we recommend you to take this online (5-hour long) course before the semester starts.
– Basic calculus, linear algebra, data structures and algorithms
