Into the land of automatic differentiation

PhD Course in Computer Science

Timetable: July 7-11, 10-12 AM.
Attendance: Aula G0, Palazzina G, Viale Regina Elena 295

Overview

This is a short (10 hours) overview to the topic of automatic differentiation, which is (among other things) the cornerstone of all deep learning frameworks. We will overview the basics of autodiff, implement from-scratch a simplified PyTorch-like API, and quickly overview some advanced material (e.g., implicit differentiation).

Material

	Material
Slides	Link
Notes	Notion page
Labs	List

Advanced reading material

• Alice’s adventures in a differentiable wonderland (Chapter 6)
• Automatic differentiation in machine learning: a survey
• The elements of differentiable programming