 |
Hongyang R. Zhang Office Address: 177 Huntington Ave 2211, Boston, MA 02115 Email Contact: ho.zhang@northeastern.edu |
I am an assistant professor at Northeastern University in the Khoury College of Computer Sciences.
Research interests
I work on a broad range of topics at the intersection of machine learning, optimization algorithms, ML theory, and networks.
On the machine learning side, we have been working on foundation models and generative models, designing new algorithms for multitask learning, supervised fine-tuning, and in-context learning. I am particularly interested in understanding how neural networks learn to extract information from data, and how this knowledge transfers to downstream tasks. To this end, we are developing a Hessian-based computational framework where we probe the second-order information such as the spectral statistics of the loss Hessian, and apply this framework to various downstream settings.
On the theory side, we have been looking into sampling complexities for learning graph neural networks and geometric data, low-rank matrix completion given little data, and multi-objective reinforcement learning.
Other topics I have recently been thinking about include algorithmic mechanism design for policy learning, formulating regularization in LLM reasoning, and time-series graph data in the context of road networks from a safety perspective.
Recent updates
Talk slides about our recent line of work on a Hessian view of supervised fine-tuning, task attribution, and RL.
New paper on efficient kernel methods for language model and data attribution.
A new paper using a self-normalization estimator for matrix completion from very sparse observations.
A satellite image dataset for traffic accident prediction and causal analysis.
A new paper on training neural networks to understand algorithmic reasoning!
A new algorithm for multi-objective and meta reinforcement learning!
Talk slides about recent work on in-context learning and a Hessian view of grokking.
New paper on a linear-time data selection algorithm for in-context learning.
An updated paper on transfer learning random matrices accepted at JMLR! We analyze a (classical) hard parameter sharing estimator and find that simply rebalancing the data leads to minimax optimal rates.
New manucript on an ensemble method for fine-tuning LLMs, built on top of LoRA.
A list of older logs.
Note for prospective students: I'm always looking for students who are interested in working with me. If you are a student at Northeastern, please feel free to contact me.
Biography
I received my Ph.D. in Computer Science from Stanford University and my B.Eng. in Computer Science from Shanghai Jiao Tong University. Subsequently, I spent a year as a postdoc within the Statistics and Data Science department at the Wharton School of the University of Pennsylvania.
Broader Impacts
I enjoy working on technically challenging problems, while striving for broader impacts by creating new knowledge that will benefit the society, as well as fostering the next generation of engineers and researchers.
I support accessible and reproducible research.