EECE 571J: Trustworthy Machine Learning (Spring 2022)

1 Overview

Machine Learning (ML) is a subfield of Artificial Intelligence where computer algorithms are learning “by example”, using past data. ML now replaces humans at many critical decision points and is used in various applications, such as banking and finance, image and speech processing, healthcare, and more. However, like traditional software, AI systems are often faulty and vulnerable to attacks. For example, Amazon had to scrap an AI-based recruiting tool that showed bias against women while Alexa and Siri were recently manipulated with hidden commands that humans cannot hear.

This seminar-style course will explore different topics in emerging research areas related to security, privacy, explainability, ethics, and fairness in machine learning. Students will learn about quality assurance methods for ML systems, attacks against ML systems, defense techniques to mitigate such attacks, and ethical implications of using ML systems.

The course assumes students already have a basic understanding of machine learning. Most of the course readings will come from both seminal and recent papers in the field. Each student will read, summarize, and present several scientific papers, as well as propose, implement, and present their own original project. As such, the course will also focus on polishing the students’ research, development, communication, and technical presentation skills. 

1.1 Learning Objectives

By the end of the course, students will learn:

• ML application scenarios and possible pitfalls;
• Quality assurance for ML systems;
• Examples of attack and defense mechanisms;
• Explainability and interpretability;
• Fairness and ethic considerations;
• Efficient technical communication and presentation skills.

1.2 Course Prerequisites

This course does not have formal prerequisites. However, previous programming experience and a basic understanding of machine learning (equivalent to CPSC 340 or ELEC400M/ELEC571M) are necessary.

2 Communication and Important Links

• The class meets on Wednesdays at 2pm-5pm PST. A Zoom link for the meeting is available in Canvas.
• Office hours are held on Tuesdays at 4:30pm-5:30pm PST. A Zoom link for the meeting is available in Canvas.
• Course syllabus, reading material, project milestones, etc. are available on Canvas.
• Grades for the assignments will be posted throughout the term. Students are encouraged to check it frequently and let the staff know of any mistakes and inconsistencies.
• Course communication (questions, discussions, announcements, etc.) will be done via Piazza. Please enroll yourself!
  Like most online services, Piazza is hosted outside Canada. Please come and talk to the instructor if you are uncomfortable using Piazza.

2.1 Support Resources

• Student Guide to Canvas at UBC
• Student Guide to Zoom at UBC
• UBC COVID-19 Information for Students
• Managing Your Mental Health During the COVID-19 Outbreak

2.2 Time Zone in Canvas

The course is set to Pacific Time (PT) time zone, where the University of British Columbia Vancouver Campus is located. All due dates are set to PT. Canvas will not automatically change time zones for you. If you want Canvas to display dates in your local time zone, you can go into your settings and adjust to your personal local time zone. Please refer to Canvas guide on how to set a time zone in your user account.

3 Topics and Tentative Schedule

4 Course Format

4.1 Reading Assignments

For weeks 4-5 and 8-12, students will read the assigned research papers (two papers each week). Each student will submit a one-page summary of each paper that describes (a) the technical approach and (b) a critical review of the paper.

For (a), describe, in bullet points, the input and outputs to the approach, its technical novelty, how the approach was evaluated, and what the results show. The description should take about 3/4 of the page. Points will be deduced for explanations that are not clear or not specific to the paper.

For (b), specify, in bullet points, 1-2 main strengths and weaknesses of the paper (not including those listed in the paper) and 1-2 suggestions for improvement and follow-up work. Points will be deduced for unclear statements and for listing non-original strength / weaknesses / suggestions, i.e., those stated in the paper.

A template MS-Word document for paper summaries can be found here.

4.2 Paper Presentations

Each week, a student will present one of the assigned research papers to the class (two students each week). The student should motivate the need for the contribution made by the paper, summarize the proposed technique and its evaluation, discuss the strengths and weaknesses of the approach (beyond those listed in the paper), and lead the discussion on the paper. Depending on the number of course participants, each student will present 1-2 papers. Students do not need to submit summaries of the papers they present.

4.3 Homework

The first and only homework assignment (HW1) is due at the beginning of class on Week 3. The students are expected to implement a simple ML classification algorithm using the Scikit-learn machine learning library, analyze its properties, and describe / demonstrate the result in class. The detailed specification for the assignment will be given in class and be posted on Piazza.

4.4 Project

The project will be performed by groups of 2-3 students. The scope of each group's project should match the number of students involved. The expectation for the project is to deepen the class’s understanding in topics related to trustworthy AI. That can include replication studies of existing techniques, novel applications of these techniques to different scenarios, collection of statistical data on existing vulnerabilities and their impact on the society, or novel literature reviews. Come to talk to the course instructor at least one week before the deadline if you want some ideas for inspiration!

There are five deliverables for the project:

• M0: Groups formed.
• M1: Project proposals.
• The proposal should be up to 2 pages and should include (a) description of the problem solved in the project, (b) proposed technical approach, (c) proposed evaluation, (d) timetable and the planned role of each team member.
• Each group should set a meeting with the course instructor to discuss their project idea before submitting the proposal.
• M2: The first project report.
• The report should be up to 5 pages and should include (a) description of the problem solved in the project, possibly refined, (b) technical approach, (c) results achieved so far and contribution of each team member, (d) plan to completion the planned role of each team member.
• M3: Demos and presentations in the course workshop.
• M4: The final project report.
• The report should be up to 8 pages and should include (a) description of the problem solved in the project, (b) explicit statement of contribution and its novelty, relation to existing work, (c) description of the technical approach, (d) evaluation and results, (e) contribution of each team member, (f) lessons learned and future research directions.

5 Grading

This course does not have a final exam. The grading is based on the following components:

• HW1 – 8%
• Summaries of the assigned research papers – 14%
• Presentation of 1-2 research papers in class – 25%
• Project – 45%, specifically:
• Project proposal summary and presentation – 5% (only if final report is submitted)
• First project report – 10% (only if final report is submitted)
• Final demo and presentation – 15%
• Final project report – 15%
• Constructive participation in class discussions – 8%

6 Absence and Late Deliverables

• This course is based on group discussions. As such, the attendance in lectures is mandatory. If you are sick or miss a class for any other justified reason, you should notify the instructor ahead of time (via Piazza).
• Late deliverables:
  • Each student is given 3 grace days per term in total, which can be used for any individual assignment(s).
  • Each team is given another 3 grace days per term in total, which can be used for milestones M0, M2, and M4.
  • Use these grace days as you wish to help manage your time, but use them wisely. You can use all three days on one deliverable, or distribute them across deliverables.
  • Grace days are counted as integers, i.e., if you are one hour late, that counts as one grace day.
  • Once all your grace days are all used, late submissions will receive a grade of zero.

7 University Policies

UBC provides resources to support student learning and to maintain healthy lifestyles but recognizes that sometimes crises arise and so there are additional resources to access including those for survivors of sexual violence. UBC values respect for the person and ideas of all members of the academic community. Harassment and discrimination are not tolerated nor is suppression of academic freedom. UBC provides appropriate accommodation for students with disabilities and for religious, spiritual and cultural observances. UBC values academic honesty and students are expected to acknowledge the ideas generated by others and to uphold the highest academic standards in all of their actions. Details of the policies and how to access support are available here.