AI504 Programming for AI

Learn and practice essential programming skills for conducting machine learning and deep learning research.

Reference Book

• https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf

Topics

1. Numpy
2. Scikit-Learn
3. PyTorch, Logistic Regression + Multilayer Perception
   • Lecture: https://youtu.be/6Q3AzGDd5oc
   • Practice: https://youtu.be/Tdz8dx8wYTE
4. Autoencoders & Denoising Autoencoders
   • Lecture: https://youtu.be/Mx2n4OtHq_s
   • Practice: https://youtu.be/J0YcBKmLfbA
5. Variational Autoencoders
   • Lecture: https://youtu.be/EErBe3-iOg4
   • Practice: https://youtu.be/YSTvAc8kjmU
6. Generative Adversarial Networks
   • Lecture: https://youtu.be/4d5vatIV21g
   • Practice: https://youtu.be/ZKOavcIfxW0
7. Convolutional Neural Networks
   • Lecture: https://youtu.be/5UxKiIThs-I
   • Practice: https://youtu.be/QKout3TEgWU
8. Word2Vec + Subword Encoding
   • Lecture: https://youtu.be/aOVBIRP2_vA
   • Practice: https://youtu.be/KBw35w3QuVQ
   • References:
     • https://github.com/rsennrich/subword-nmt
     • http://www.statmt.org/moses/?n=Moses.Baseline
     • http://data.statmt.org/wmt17_systems/training/
9. Recurrent Neural Network + Sequence-to-Sequence
   • Lecture: https://youtu.be/R6PDzmMOHXI
   • Practice: https://youtu.be/FglgBa7rc_g
10. Image-To-Text
    • Lecture: https://youtu.be/NoWMhfFI51c
    • Practice: https://youtu.be/G7IBb1T2IL4
11. Transformers
    • Lecture: https://youtu.be/h8avp8yDKV4
    • Practice: https://youtu.be/7lJgMQQoCg8
12. BERT (& GPT)
    • Lecture: https://youtu.be/OFUYHRbHqOk
    • Practice: https://youtu.be/HS94-5QPWt4
13. Graph Neural Networks
    • Lecture: https://youtu.be/RRGS9jBBg_M
    • Practice: https://youtu.be/tr7nq-4WfEA
14. Neural Ordinary Differential Equations
    • Lecture: https://youtu.be/sIFnARdTVvE
    • Practice: https://youtu.be/0oSayyDp2M0

General Requirements

• Python 3
• jupyter notebook/ jupyterlab
• Numpy
• Torch
• Torch Vision
• Sklearn
• Matplotlib

Papers

• https://arxiv.org/pdf/1806.09055.pdf
• https://arxiv.org/pdf/1706.03762.pdf
• https://arxiv.org/pdf/2005.14165.pdf
• https://arxiv.org/abs/1310.4546
• https://deepmind.com/research/publications/mastering-game-go-deep-neural-networks-tree-search
• https://proceedings.neurips.cc/paper/2017/file/2650d6089a6d640c5e85b2b88265dc2b-Paper.pdf

Credit to: Associate Prof Edward Choi, KAIST