Course Name: Machine Learning (ML)
Participants: BSc Mathematics and Data Science (L3) students
Institution: Sorbonne University
Instructor: Dr. Tanujit Chakraborty
Timeline: January to May
Total Teaching: 60 Sessions
Email: ctanujit@gmail.com
The Data Science folder contains the data and code used during class demonstrations.
The Machine Learning folder contains the slides used in class.
Tutorials (Exercises with Solutions) are uploaded as a PDF (Pen and Paper Exercises on Machine Learning.pdf).
Course Objectives: The aim of the course is to introduce students to some of the fundamental tools of Data Science and Machine Learning. The course is ideally divided into three parts. In the first part, supervised learning methods for prediction and classification are presented. In particular, it covers linear regression, kernel methods, decision trees, and support vector machines. The second part of the course presents unsupervised learning methods, including clustering and spectral PCA. The third and final part of the course introduces deep learning (neural networks and deep neural networks).
Course Syllabus: Topics to be covered in this course include
- Introduction to Machine Learning
- Essential Calculus and Linear Algebra
- Optimization Techniques for ML
- Probability Essentials and Parameter Estimation
- MAP Estimation and Bayesian Inference
- Basics of Python
- Learning with Prototypes and K-Nearest Neighbors
- Learning using Decision Trees
- Introduction to Linear Models and Linear Regression
- Probabilistic Linear Regression
- Logistic Regression
- Generative Models for Supervised Learning
- Perceptron and Support Vector Machines
- Nonlinear Learning with Kernels
- Unsupervised Learning using K-Means Algorithm
- Hierarchical and Graph Clustering
- Principal Component Analysis for Dimensionality Reduction
- Nonlinear dimensionality reduction: Kernel PCA and Manifold Learning
- EM Algorithm and Latent Variable Models
- Neural Networks: MLP and Backpropagation
- Introduction to Deep Neural Networks
- Bias-Variance Trade-off and Ensemble Learning
- Reinforcement Learning
- Semi-supervised Learning and Multi-task Learning
- Recommendation Systems
Evaluation Components: The evaluation components for the Machine Learning course will be as follows:
- Mid-Term Exam - 25%; 2) Project Work/Kaggle Competition - 25%; 3) End-Term Test - 50%.
Textbooks and References:
This course will be taught using slides, lecture notes, tutorials, papers, and Python notebooks. Recommended books are listed below (*indicates the most-used books in this course). For students, the book "Dive into Deep Learning" covers many topics in Machine Learning and deep learning and includes coding from scratch in Python (https://d2l.ai/index.html).
- Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for machine learning. Cambridge University Press. (Read Online)
- Friedman, J., Hastie, T., & Tibshirani, R. (2017). The Elements of Statistical Learning. Springer series in statistics. (Read Online)
- Bishop, C. M. (2006). Pattern recognition and machine learning*. Springer. (Read Online)
- Murphy, K. P. (2012). Machine learning: a probabilistic perspective*. MIT Press. (Read Online)
- Shalev-Shwartz, S., & Ben-David, S. (2014). Understanding machine learning: From theory to algorithms. Cambridge University Press. (Read Online)
- Hart, P. E., Stork, D. G., & Duda, R. O. (2000). Pattern classification. Hoboken: Wiley. (Read Online)
- Daumé, H. (2017). A course in machine learning. (Read Online)
- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning*. MIT Press. (Read Online)
- Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2018). Foundations of machine learning. MIT Press. (Read Online)
- Zhang, A., Lipton, Z. C., Li, M., & Smola, A. J. (2021). Dive into deep learning. (Read Online)
Some Interesting Papers / Webpages For Reading:
• Machine Learning - Machine learning: Trends, perspectives, and prospects Paper in Science (2015): https://www.cs.cmu.edu/~tom/pubs/Science-ML-2015.pdf
• Deep Learning - Deep Learning Review Paper in Nature (2015): https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf
• Math Refresher - Mathematics for Machine Learning (2018): https://gwthomas.github.io/docs/math4ml.pdf