Tiny Machine Learning (COMPSCI 249R)

Tiny Machine Learning (COMPSCI 249R)

Fall 2023 | Harvard University
Tiny machine learning (TinyML) is defined as a fast-growing field of machine learning technologies and applications including hardware (dedicated integrated circuits), algorithms and software capable of performing on-device sensor (vision, audio, IMU, biomedical, etc.) data analytics at extremely low power, typically in the mW range and below, and hence enabling a variety of always-on use-cases and targeting battery-operated devices. The pervasiveness of ultra-low-power embedded devices, coupled with the introduction of embedded machine learning frameworks like TensorFlow Lite for Microcontrollers, will enable the mass proliferation of AI-powered IoT devices. The explosive growth in machine learning and the ease of use of platforms like TensorFlow (TF) make it an indispensable topic of study for modern computer science and electrical engineering students.

General

Course Topics

The course provides a sweeping overview of machine learning systems, from foundational concepts like the stages of machine learning to advanced topics such as hardware acceleration and on-edge generative AI. This includes a journey through data engineering, optimized model frameworks, and sustainability dimensions of ML, all tailored to embedded environments.
Overview and Introduction to Embedded Machine Learning
Data Engineering
Embedded Machine Learning Frameworks
Efficient Model Representation and Compression
Performance Metrics and Benchmarking of ML Systems
Learning on the Edge
Hardware Acceleration for Edge ML: GPUs, TPUs and FPGAs
Embedded MLOps
Secure and Privacy-Preserving On-Device ML
Responsible AI
Sustainability at the Edge
Generative AI at the Edge

Textbook

The field of embedded machine learning systems is rapidly evolving. While there is a TinyML textbook available, it has become somewhat dated. We'll primarily rely on the latest academic publications for our studies. You may use the TinyML textbook as a reference material when needed. We will be drawing some of the content from the textbook titled “AI at the Edge,” which contains more up-to-date methods and examples. We can make an online version available to you via Canvas that cannot be distributed outside of class. However, we encourage students to get their own copy so that it is easier to follow and work through the material.

Reading and Lecture Notes

Lecture 1: Overview and Introduction to TinyML

Lecture 2: Data Engineering

Required Readings
Optional Readings

Lecture 3: Machine Learning Frameworks

Required Readings
Optional Readings

Lecture 4: Model Optimizations

Required Readings
Optional Readings

Lecture 5: Learning on the Edge

Required Readings
Optional Readings

Lecture 6: Acceleration for ML: GPUs, TPUs and FPGAs

Required Readings
Optional Readings

Lecture 7: MLOps

Required Readings
Optional Readings
Data Cascades in High-Stakes AI

Secure and Privacy-Preserving On-Device ML

Required Readings
Optional Readings

Lecture 9: Responsible AI

Required Readings
Optional Readings

Lecture 10: Sustainability at the Edge

Required Readings
Optional Readings

Lecture 11: Generative AI at the Edge

Required Readings
Optional Readings

Additional Lecture Note

Assignments