Welcome to my site & blog! I'm a Computer Science & Statistics student at the University of Waterloo interested in machine learning and programming languages.

Previously, I did deep learning research on collaborative self-driving at Uber ATG, helped improve the neural network optimization algorithm library K-FAC at Google Brain, accelerated BERT inference at NVIDIA TensorRT, quantitative research on alternative data at Citadel, and developed data-driven models to identify fraud at John Hancock Financial. I've also worked as a Research Assistant for Prof. Gautam Kamath in differential privacy, Prof. Lin Tan in deep learning-driven software analysis, and Prof. Pascal Poupart on neural network parameter learning.

Follow me on twitter, I mostly tweet about technical topics (though not so active recently...)! Checkout my resources page for links to cool blog posts, talks, and other gems. Feel free to reach out through email: nicholas.vadivelu [at] gmail [dot] com.

News

Jan 2022	Runner-Up for the CRA Outstanding Undergraduate Research Award [award page]
May 2021	Awarded the 2021 Jessie W.H. Zou Memorial Award. [article]
May 2021	Open sourced my implementation of Progressively Growing GANs in Flax. [GitHub]
May 2021	Released JAX ResNet - Implementations and checkpoints of ResNet variants in Flax [GitHub] [PyPI]
Feb 2021	[FINISHED] I'm currently running a course-group based on Bayesian Data Analysis by Aki Vehtari.   Join us! [Discord] [problem solutions/demos]
Jan 2021	Released ShapeCheck - A framework-agnostic library for checking array/tensor shapes at runtime. [GitHub] [PyPI]
Dec 2020	Released Awesome JAX - A curated list of resources. [GitHub]

Papers

(* = equal contribution)

• Learning to Communicate and Correct Pose Errors. Nicholas Vadivelu, Mengye Ren, James Tu, Jingkang Wang, Raquel Urtasun. Conference on Robot Learning (CoRL), Virtual, 2020. [arxiv] [abstract] [video]
  
  Learned communication makes multi-agent systems more effective by aggregating distributed information. However, it also exposes individual agents to the threat of erroneous messages they might receive. In this paper, we study the setting proposed in V2VNet, where nearby self-driving vehicles jointly perform object detection and motion forecasting in a cooperative manner. Despite a huge performance boost when the agents solve the task together, the gain is quickly diminished in the presence of pose noise since the communication relies on spatial transformations. Hence, we propose a novel neural reasoning framework that learns to communicate, to estimate potential errors, and finally, to reach a consensus about those errors. Experiments confirm that our proposed framework significantly improves the robustness of multi-agent self-driving perception and motion forecasting systems under realistic and severe localization noise.
  
  
• Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorization. Pranav Subramani*, Nicholas Vadivelu*, Gautam Kamath. NeuRIPS Privacy-Preserving Machine Learning Workshop, Virtual, 2020. [arxiv] [code] [abstract]
  
  A common pain point in differentially private machine learning is the significant runtime overhead incurred when executing Differentially Private Stochastic Gradient Descent (DPSGD), which may be as large as two orders of magnitude. We thoroughly demonstrate that by exploiting powerful language primitives, including vectorization, just-in-time compilation, and static graph optimization, one can dramatically reduce these overheads, in many cases nearly matching the best non-private running times. These gains are realized in two frameworks: JAX and TensorFlow. JAX provides rich support for these primitives as core features of the language through the XLA compiler. We also rebuild core parts of TensorFlow Privacy, integrating features from TensorFlow 2 as well as XLA compilation, granting significant memory and runtime improvements over the current release version. These approaches allow us to achieve up to 50x speedups in comparison to the best alternatives.
  
  

Blog Posts

Talks

• Optimizing Machine Learning Code in NumPy & SciPy University of Waterloo Data Science Club, Waterloo, Mar 2022. [video] [description] [code] [slides]
  
  As data and problems get bigger, implementing numerical algorithms efficiently is crucial for the modern data scientist to keep up. NumPy and SciPy are among the most common data science libraries to write fast numerical algorithms, but it's easy to use them in a sub-optimal way. In this workshop, we'll go over a case-study of writing, benchmarking, and optimizing a machine learning algorithm using NumPy and SciPy. The target audience is intermediate NumPy & SciPy users looking to level up their game.
  
  
• Contributing to Open Source Data Science Libraries University of Waterloo Data Science Club, Waterloo, Jan 2022. [video] [description] [slides]
  
  Contributing to open source is one of the best ways to improve your programming skills, get involved in the data science community, and show off your skills to employers. In this workshop, we'll go over the entire process of opening issues, setting up your environment, adding new code to libraries, integrating tests, and creating pull requests. Most of this workshop will be a live demo on a real data science library (Optax). Basic knowledge of Python/Git is expected (syntax, installing libraries, etc).
  
  
• Clustering for Image Analysis (with Kanika Chopra) WiSTEM High School Student Conference, Virtual, Feb 2021. [description] [slides] [notebook]
  
  Learn about the basics of K-Means clustering as demonstrated on the MNIST dataset. We cover a basic introduction to Data Science & Machine Learning, Python & NumPy, and finally, implement K-Means.
  
  
• Establishing a Productive Machine Learning Workflow Hack the North++, Virtual, Jan 2021. [video] [description] [code] [slides] [syllabus] [hackpack]
  
  Conventional software engineering workflows are sub-optimal for DS & ML. Through this workshop, you'll learn about best practices for experiment configuration, productivity tools, continuous integration for data science, static analysis, and more.
  
  
• Interactive Data Visualization with Altair Hack the North++, Virtual, Jan 2021. [video] [description] [notebook] [syllabus] [hackpack]
  
  Learn how to create beautiful, interactive data visualizations using Altair in Python. Through this workshop, we'll explore data using Altair, learn about the grammar of graphics, create simple dashboards, and deploy them for others to use.
  
  
• Overview of Data Science and Data Science Careers. University of Waterloo Data Science Club, Waterloo, Aug 2020. [video] [description] [slides]
  
  In this talk, you'll learn about UWaterloo's data science club, data science in general, data science careers, and how to tailor your education for a data science career. This presentation was made for first-year students entering UWaterloo in Fall 2020.
  
  
• What You See is What You Get: Exploiting Visibility for 3D Object Detection. Uber ATG Paper Reading Group, Toronto, July 2020. [description] [slides]
  
  Summary and critique of the paper What You See is What You Get: Exploiting Visibility for 3D Object Detection by Peiyun Hu, Jason Ziglar, David Held, Deva Ramanan.
  
  
• Introduction to JAX for Machine Learning and More. University of Waterloo Data Science Club, Waterloo, July 2020. [video] [description] [notebook]
  
  JAX is a numeric computing library with a numpy-like interface. It specializes in function transformations such as auto-differentiation, just in time compilation (via XLA), and more. Through this workshop, you'll learn about what JAX is, how to use it, how it is different from other libraries like PyTorch and TensorFlow, why you should consider adding it to your toolkit.
  
  
• Stand-Alone Self-Attention in Vision Models. Uber ATG Paper Reading Group, Toronto, April 2020. [description] [slides]
  
  Summary and critique of the paper Stand-Alone Self-Attention in Vision Models by Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello, Anselm Levskaya, Jonathon Shlens.
  
  
• Neural Network Optimization Methods Reading Group, Waterloo, December 2019. [description] [slides]
  
  Overview of gradient-based neural network optimization algorithms, including momentum, nesterov, adagrad variants, adam variants, and K-FAC.
  
  
• Introduction to Neural Networks in TensorFlow 2.0. Laurier Developer Student Club, Waterloo, Nov 2019. [video] [description] [notebook]
  
  This workshop introduces neural networks in TensorFlow 2.0. Topics covered include linear regression, multiclass logistic regression, gradient descent, and neural network. TF 2.0 features such as tf.keras, tf.data, and more are covered. This was co-hosted by the UW Data Science Club and Laurier Developer Student Club.
  
  
• Introduction to Machine Learning with Scikit-learn. Hack the North, Waterloo, Sep 2019. [video] [description] [notebook] [slides]
  
  Through this workshop you will learn how to quickly model and understand datasets using scikit-learn. Topics will include a basic introduction to using decisions trees and random forests, understanding feature importance, identifying model weaknesses, explaining your model and more. This workshop will focus on evaluating your model with less emphasis on syntax and mechanics.
  
  
• Introduction to Pandas for Python. Hack the North, Waterloo, Sep 2019. [video] [description] [notebook]
  
  Through this workshop, you will learn how to use Pandas to explore and “wrangle” datasets. Topics will include an introduction to Jupyer Notebooks/Colab, data cleaning with pandas, feature engineering with pandas, basic visualization and more. This workshop will focus on actual coding.