PhD Course in Data Science
Attendance: Zoom link, contact me if you do not receive the passcode.
Building a deep learning model is a complex task, full of interacting design decisions, data engineering, parameter tweaking, and experimentation. Having access to powerful tools for versioning, storing, and analyzing every step of the process (MLOps) is essential.
The aim of this practical course is to start from a simple deep learning model implemented in a notebook, and port it to a ‘reproducible’ world by including code versioning (Git), data versioning (DVC), experiment logging (Weight & Biases), hyper-parameter tuning, configuration (Hydra), and ‘Dockerization’. While the focus is on vertical, well-established tools, we will discuss more advanced integrated frameworks (e.g., MLFlow) and techniques (e.g., CI/CD pipelines).
Setup your machine
We will install most libraries as we go along. For the initial setup, perform an Anaconda installation on your machine, and create an environment:
Then, install a few generic prerequisites (notebook handling, Pandas, …):
Organization of the material
Slides will be available below, while the code will be uploaded to a GitHub repository. The course is split into exercises (Git, DVC, …). To start the exercise, swith to the corresponding Git branch, and follow the instructions on video or in the corresponding README file. To see the completed exercise, you can switch to a completed branch, as shown below.
To follow the DVC exercise, check in the table below the name of the branch (exercise2_dvc), and perform a checkout:
If you want to see the completed exercise, add _completed to the name of the branch:
You can inspect the commits to look at specific changes in the code:
If you want to inspect a specific change, you can checkout again using the ID of the commit.
|0||Introduction||-||Slides, Bare repository, Video|
|1||Deep learning recap||-||Notebook, Video|
|2||Git & Scripting||exercise1_git||Slides, Video, Code|
|3||Hydra configuration||exercise2_hydra||Hydra repository, Video, Code|
|4||Data versioning with DVC||exercise3_dvc||DVC Website, Slides, Video (part 1), Video (part 2), Code|
|5||Docker||exercise4_docker||Docker Website, Slides, Video, Code|
|6||Weight & Biases||exercise5_wandb||Weights & Biases Website, Video, Code|
|7||Continuous integration||exercise6_hooks||Video, Code|
Advanced reading material
The new edition of Full Stack Deep Learning (UC Berkeley CS194-080) covers a larger set of material than this course.