EECE 571J: Trustworthy Machine Learning (Spring 2022)

Contents

  1. Overview
  2. Communication and Links
  3. Topics and Tentative Schedule
  4. Course Format
  5. Grading
  6. Absence and Late Deliverables
  7. University Policies

1 Overview

Machine Learning (ML) is a subfield of Artificial Intelligence where computer algorithms are learning “by example”, using past data. ML now replaces humans at many critical decision points and is used in various applications, such as banking and finance, image and speech processing, healthcare, and more. However, like traditional software, AI systems are often faulty and vulnerable to attacks. For example, Amazon had to scrap an AI-based recruiting tool that showed bias against women while Alexa and Siri were recently manipulated with hidden commands that humans cannot hear.

This seminar-style course will explore different topics in emerging research areas related to security, privacy, explainability, ethics, and fairness in machine learning. Students will learn about quality assurance methods for ML systems, attacks against ML systems, defense techniques to mitigate such attacks, and ethical implications of using ML systems.

The course assumes students already have a basic understanding of machine learning. Most of the course readings will come from both seminal and recent papers in the field. Each student will read, summarize, and present several scientific papers, as well as propose, implement, and present their own original project. As such, the course will also focus on polishing the students’ research, development, communication, and technical presentation skills. 

1.1 Learning Objectives

By the end of the course, students will learn:

1.2 Course Prerequisites

This course does not have formal prerequisites. However, previous programming experience and a basic understanding of machine learning (equivalent to CPSC 340 or ELEC400M/ELEC571M) are necessary.

2 Communication and Important Links

2.1 Support Resources

2.2 Time Zone in Canvas

The course is set to Pacific Time (PT) time zone, where the University of British Columbia Vancouver Campus is located. All due dates are set to PT. Canvas will not automatically change time zones for you. If you want Canvas to display dates in your local time zone, you can go into your settings and adjust to your personal local time zone. Please refer to Canvas guide on how to set a time zone in your user account.

3 Topics and Tentative Schedule

Week

Topic

Major Deadlines
(11pm on Tuesday, day before the class, unless stated otherwise)

W1: Jan 12

Introductions; ML application scenarios; what can go wrong; Course logistics

W2: Jan 19

AI Trustworthiness - Overview

Submit 2-3 discussion points for each video

W3: Jan 26

HW1 presentations

HW1

Jan 24-Jan 25, 11pm: select papers you would like to present

W4: Feb 2

Adversarial Robustness

Project M0 (finalize groups and discuss project ideas)

W5: Feb 9

Adversarial Robustness in Software Systems

 

W6: Feb 16

Project proposal presentations

Project M1 (proposal)

W7: Feb 23 Mid-term Break  

W8: Mar 2

Explainability and Interpretability

 

W9: Mar 9

Privacy

 

W10: Mar 16

Fairness, Ethics, and Law

Project M2 (first project report)  

W11: Mar 23

Guest Lecture: Dr. Ece Kamar

 

W12: Mar 30

Industrial Perspectives

 

W13: Apr 6

Workshop:
project presentations and demos

Project M3 (presentations and demos)

(W14: Apr 13)

 

 

(W15: Apr 20)

 

M4: Final project report

(W16: Apr 27)    

4 Course Format

4.1 Reading Assignments

For weeks 4-5 and 8-12, students will read the assigned research papers (two papers each week). Each student will submit a one-page summary of each paper that describes (a) the technical approach and (b) a critical review of the paper.

For (a), describe, in bullet points, the input and outputs to the approach, its technical novelty, how the approach was evaluated, and what the results show. The description should take about 3/4 of the page. Points will be deduced for explanations that are not clear or not specific to the paper.

For (b), specify, in bullet points, 1-2 main strengths and weaknesses of the paper (not including those listed in the paper) and 1-2 suggestions for improvement and follow-up work. Points will be deduced for unclear statements and for listing non-original strength / weaknesses / suggestions, i.e., those stated in the paper.

A template MS-Word document for paper summaries can be found here.

4.2 Paper Presentations

Each week, a student will present one of the assigned research papers to the class (two students each week). The student should motivate the need for the contribution made by the paper, summarize the proposed technique and its evaluation, discuss the strengths and weaknesses of the approach (beyond those listed in the paper), and lead the discussion on the paper. Depending on the number of course participants, each student will present 1-2 papers. Students do not need to submit summaries of the papers they present.

4.3 Homework

The first and only homework assignment (HW1) is due at the beginning of class on Week 3. The students are expected to implement a simple ML classification algorithm using the Scikit-learn machine learning library, analyze its properties, and describe / demonstrate the result in class. The detailed specification for the assignment will be given in class and be posted on Piazza.

4.4 Project

The project will be performed by groups of 2-3 students. The scope of each group's project should match the number of students involved. The expectation for the project is to deepen the class’s understanding in topics related to trustworthy AI. That can include replication studies of existing techniques, novel applications of these techniques to different scenarios, collection of statistical data on existing vulnerabilities and their impact on the society, or novel literature reviews. Come to talk to the course instructor at least one week before the deadline if you want some ideas for inspiration!

There are five deliverables for the project:

5 Grading

This course does not have a final exam. The grading is based on the following components:

6 Absence and Late Deliverables

7 University Policies

UBC provides resources to support student learning and to maintain healthy lifestyles but recognizes that sometimes crises arise and so there are additional resources to access including those for survivors of sexual violence. UBC values respect for the person and ideas of all members of the academic community. Harassment and discrimination are not tolerated nor is suppression of academic freedom. UBC provides appropriate accommodation for students with disabilities and for religious, spiritual and cultural observances. UBC values academic honesty and students are expected to acknowledge the ideas generated by others and to uphold the highest academic standards in all of their actions. Details of the policies and how to access support are available here.