Fall 2023 | Harvard University
Tiny machine learning (TinyML) is defined as a fast-growing field of machine learning technologies and applications including hardware (dedicated integrated circuits), algorithms and software capable of performing on-device sensor (vision, audio, IMU, biomedical, etc.) data analytics at extremely low power, typically in the mW range and below, and hence enabling a variety of always-on use-cases and targeting battery-operated devices. The pervasiveness of ultra-low-power embedded devices, coupled with the introduction of embedded machine learning frameworks like TensorFlow Lite for Microcontrollers, will enable the mass proliferation of AI-powered IoT devices. The explosive growth in machine learning and the ease of use of platforms like TensorFlow (TF) make it an indispensable topic of study for modern computer science and electrical engineering students.
General
Course Topics
The course provides a sweeping overview of machine learning systems, from foundational
concepts like the stages of machine learning to advanced topics such as hardware acceleration
and on-edge generative AI. This includes a journey through data engineering, optimized model
frameworks, and sustainability dimensions of ML, all tailored to embedded environments.
Overview and Introduction to Embedded Machine Learning
Data Engineering
Embedded Machine Learning Frameworks
Efficient Model Representation and Compression
Performance Metrics and Benchmarking of ML Systems
Learning on the Edge
Hardware Acceleration for Edge ML: GPUs, TPUs and FPGAs
Embedded MLOps
Secure and Privacy-Preserving On-Device ML
Responsible AI
Sustainability at the Edge
Generative AI at the Edge
Textbook
The field of embedded machine learning systems is rapidly evolving. While there is a TinyML textbook available, it has become somewhat dated. We'll primarily rely on the latest academic
publications for our studies. You may use the TinyML textbook as a reference material when
needed. We will be drawing some of the content from the textbook titled “AI at the Edge,”
which contains more up-to-date methods and examples. We can make an online version
available to you via Canvas that cannot be distributed outside of class. However, we encourage
students to get their own copy so that it is easier to follow and work through the material.
Reading and Lecture Notes
Lecture 1: Overview and Introduction to TinyML
Lecture 2: Data Engineering
Required Readings
Optional Readings
Lecture 3: Machine Learning Frameworks
Required Readings
Optional Readings
Lecture 4: Model Optimizations
Required Readings
Optional Readings
Lecture 5: Learning on the Edge
Required Readings
Optional Readings
Lecture 6: Acceleration for ML: GPUs, TPUs and FPGAs
Required Readings
Optional Readings
Lecture 7: MLOps
Required Readings
Optional Readings
Data Cascades in High-Stakes AI
Secure and Privacy-Preserving On-Device ML
Required Readings
Optional Readings
Lecture 9: Responsible AI
Required Readings
Optional Readings
Lecture 10: Sustainability at the Edge
Required Readings
Optional Readings
Lecture 11: Generative AI at the Edge
Required Readings
Optional Readings