Course: Graph Machine Learning

The highest activity a human being can attain is learning for understanding,
because to understand is to be free. Baruch Spinoza

Lecturer
Zahra Taheri	Data Science Center Shahid Beheshti University Winter 2023

💡 Course Overview

Graph Machine Learning is a course that focuses on the application of machine learning algorithms on graph-structured data. Some of the key topics that are covered in the course include graph representation learning and graph neural networks, algorithms for the world wide web, reasoning over knowledge graphs, and social network analysis. The course is designed for graduate students with a background in machine learning and/or data science who want to expand their skills to work with graph data. The course may also be useful for students and professionals working in fields such as computer science, biology, chemistry, and physics that require the analysis of graph-structured data. The objective of the course is to provide students with a comprehensive understanding of graph machine learning and its various applications, challenges, and opportunities, as well as hands-on experience in implementing these algorithms.

🔰 Prerequisites

• Familiarity with the basic probability theory, and the basic linear algebra
• Basic knowledge of machine learning and/or deep learning concepts
• Familiarity with the basics of Python programming language
• Familiarity with PyTorch is a plus

📚 Recommended Materials

Books

• Graph Representation Learning by William L. Hamilton
• Network Science by Albert-László Barabási
• Networks, Crowds, and Markets: Reasoning About a Highly Connected World by David Easley and Jon Kleinberg
• Analysis of Biological Networks, 2007.

Graph Machine Learning Tools

• DGL (Deep Graph Library)
• PyG (PyTorch Geometric)
• NetworkX

Courses

• CS224W: Machine Learning with Graphs by Jure Leskovec

Tips and Tools for Data Science

• Fundamental and Useful Tools and Tips for Data Science
• Essential Steps to Set Up Your PC for Graph Machine Learning with PyG

📕 Other Materials

Books

• Deep Learning on Graphs by Yao Ma and Jiliang Tang
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges by Michael M. Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković

📖 Contents

The contents and materials related to the course will be posted here.

1. Introduction to Graph Machine Learning

Required Reading:

• Slide: Introduction; Machine Learning for Graphs by Jure Leskovec

Suggested Reading:

• Papers: To fully understand the following papers, you should be familiar with graph neural networks.
  • Node-Level Graph ML Task:
    • Alphafold2: Highly accurate protein structure prediction with AlphaFold by Jumper, et.al., Nature 2021.
  • Edge-Level Graph ML Task:
    • Graph Convolutional Neural Networks for Web-Scale Recommender Systems by Ying et.al., KDD 2018.
    • Modeling Polypharmacy Side Effects with Graph Convolutional Networks by Zitnik et.al., Bioinformatics 2018.
  • Subgraph-Level Graph ML Task:
    • ETA Prediction with Graph Neural Networks in Google Maps by Derrow-Pinion et.al., CIKM 2021.
  • Graph-Level Graph ML Task:
    • A Deep Learning Approach to Antibiotic Discovery by Stokes et.al., Cell 2020.
    • Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation by You et.al., NeurIPS 2018.
    • Learning to simulate complex physics with graph networks by Sanchez-Gonzalez et al., ICML 2020.

2. Traditional Methods for Machine Learning on Graphs

Required Reading:

• Slide: Traditional Methods for ML on Graphs by Jure Leskovec

Suggested Reading:

• Blog:

  • Link Prediction in Large-Scale Networks by Cdiscount Data Science
  • Expressive power of graph neural networks and the Weisfeiler-Lehman test by Michael Bronstein

• Slide:

  • Introduction to Kernel Methods
  • Elements of Positive Definite Kernel and Reproducing Kernel Hilbert Space

• Paper:

  • Graph Kernels by Vishwanathan et al., JMLR 2010.
  • Efficient graphlet kernels for large graph comparison by Shervashidze et al., JMLR 2009.
  • Weisfeiler-Lehman Graph Kernels by Shervashidze et al., JMLR 2011.

• Question: How does the dot product determine similarity?

3. Node Embeddings

Required Reading:

• Slide: Node Embeddings by Jure Leskovec
• Example of node2vec: Detailed Example and Implementation by Zahra Taheri

Suggested Reading:

• Blog:
  • Complete guide to understanding Node2Vec algorithm by Tomaz Bratanic
  • Node2Vec Explained by Vatsal
  • node2vec: Embeddings for Graph Data by Elior Cohen
  • Node2vec explained graphically by Remy Lau
  • Word2Vec Tutorial - The Skip-Gram Model by Chris McCormick
  • How to Use Negative Sampling With Word2Vec Model? by Vijaysinh Lendave
  • Understanding Representation Learning With Autoencoder by Nilesh Barla
• Video:
  • Graph Embeddings (node2vec) explained - How nodes get mapped to vectors by Philipp Brunenberg
  • Node2Vec: Scalable Feature Learning for Networks | ML with Graphs (Research Paper Walkthrough) by TechViz-The Data Science Guy

4. Graph Neural Networks 1:

Required Reading:

• Slide: Graph Neural Networks 1: GNN Model by Jure Leskovec

Suggested Reading:

• Blog:
  • A Gentle Introduction to Graph Neural Networks
  • Understanding Convolutions on Graphs

5. Graph Neural Networks 2:

Required Reading:

• Slide: Graph Neural Networks 2: Design Space by Jure Leskovec

Suggested Reading:

• Blog:
  • Best Graph Neural Network architectures: GCN, GAT, MPNN and more

📝 Homework and assignment

More information about homeworks, assignments, and projects will be posted here.

• Assignment Set 1: Deadline 21 Feb 2023 (2 Esfand 1401) at 11:59pm.
• Assignment Set 2: Deadline 17 Mar 2023 (26 Esfand 1401) at 11:59pm.
• Assignment Set 3: Deadline 4 Apr 2023 (15 Farvardin 1402) at 11:59pm.
• Assignment Set 4: Deadline 25 Apr 2023 (5 Ordibehesht 1402) at 11:59pm.
• Assignment Set 5: Deadline 28 Apr 2023 (8 Ordibehesht 1402) at 11:59pm.
• Assignment Set 6: Deadline 5 May 2023 (15 Ordibehesht 1402) at 11:59pm.

©️ Honor Code and Submission Policy

The Honor Code and Submission Policy are the foundation for ethical and academic standards in the Graph Machine Learning course. All students are expected to adhere to the Honor Code by not engaging in academic misconduct such as plagiarism or cheating on exams. The Submission Policy requires that all assignments are submitted on time, in the specified format, and accurately reflect the student's own work. Late submissions may be accepted with a penalty, as outlined in the policy. Failure to comply with the Honor Code and Submission Policy may result in consequences such as a reduced grade or failure in the course. It is the responsibility of all students to familiarize themselves with the Honor Code and Submission Policy and to maintain the highest level of academic integrity.

📊 Grading

The weighting scheme of the grading:

• 20% on Homeworks
• 25% on the Mid-Term Exam
• 30% on the Final Exam
• 25% on the Final Project
• Course participation and contribution in the discussions as extra credit

🕜 Course Schedule

Sunday and Tuesday 1:30pm to 3pm

Office Hours
Sunday and Tuesday 12:30pm
Also, students may ask their questions via the group of the course.

⏰ Exam

• Mid-Term: Sunday, 18 April 2023 (29 Farvardin 1402)
• Final: Saturday, 17 June 2023 (27 Khordad 1402)