Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models with Adway Girish, Alliot Nagle, Marco Bondaschi, Michael C. Gastpar, and Hyeji Kim NeurIPS, 2024
[Slides]
Transformers on Markov Data: Constant Depth Suffices with Nived Rajaraman, Marco Bondaschi, Kannan Ramachandran, and Michael C. Gastpar NeurIPS, 2024
Local to Global: Learning Dynamics and Effect of Initialization for Transformers with Marco Bondaschi, Chanakya Ekbote, Adway Girish, Alliot Nagle, Hyeji Kim, and Michael C. Gastpar NeurIPS, 2024
Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chains with Marco Bondaschi, Adway Girish, Alliot Nagle, Martin Jaggi, Hyeji Kim, and Michael C. Gastpar Mechanistic Interpretability Workshop, ICML 2024 & under review at ICLR 2025
Optimal transport mapping via input convex neural networks [Slides] with Amirhossein Taghvaei, Jason Lee, and Sewoong Oh ICML, 2020
Learning in Gated Neural Networks [Slides] with Sreeram Kannan, Sewoong Oh, and Pramod Viswanath AISTATS, 2020
Breaking the gridlock in Mixture-of-Experts: Consistent and Efficient Algorithms with Sewoong Oh, Sreeram Kannan, and Pramod Viswanath ICML, 2019
Learning One-hidden-layer Neural Networks under General Input Distributions with Weihao Gao, Sewoong Oh, and Pramod Viswanath AISTATS, 2019
LASER: Linear Compression in Wireless Distributed Optimization [Slides] with Marco Bondaschi, Thijs Vogels, Martin Jaggi, Hyeji Kim, and Michael C. Gastpar NeurIPS Federated Learning Workshop, 2023 ICML, 2024
CRISP: Curriculum based Sequential Neural Decoders for Polar Code Family with S Ashwin Hebbar, Viraj Nadkarni, Suma Bhat, Sewoong Oh, and Pramod Viswanath ICML, 2023
Machine Learning-Aided Efficient Decoding of Reed-Muller Subcodes with Mohammad Vahid Jamali, Xiyang Liu, Hessam Mahdavifar, Sewoong Oh, and Pramod Viswanath IEEE Journal on Selected Topics in Information Theory (JSAIT), 2023
TinyTurbo: Efficient Turbo Decoders on Edge with S Ashwin Hebbar, Rajesh K Mishra, Sravan Kumar Ankireddy, Hyeji Kim, and Pramod Viswanath ISIT, 2022
KO codes: Inventing Nonlinear Encoding and Decoding for Reliable Wireless Communication via Deep-learning [Slides] with Xiyang Liu, Mohammad Vahid Jamali, Hessam Mahdavifar, Sewoong Oh, and Pramod Viswanath ICML, 2021
Reed-Muller Subcodes: Machine Learning-Aided Design of Efficient Soft Recursive Decoding with Mohammad Vahid Jamali, Xiyang Liu, Hessam Mahdavifar, Sewoong Oh, and Pramod Viswanath ISIT, 2021
Barracuda: The Power of l-polling in Proof-of-Stake Blockchains with Ranvir Rana, Jiantao Jao, Sewoong Oh, Giulia Fanti, and Pramod Viswanath ACM MobiHoc, 2019 (Best paper award)
Equivalence of additive-combinatorial linear inequalities for Shannon entropy and differential entropy with Yihong Wu IEEE Transactions on Information Theory, 2018
On additive-combinatorial affine inequalities for Shannon entropy and differential entropy with Yihong Wu IEEE International Symposium on Information Theory (ISIT), 2016
Event-driven stochastic approximation with Vivek S. Borkar and Neeraja Sahasrabudhe Indian Journal of Pure and Applied Mathematics, June 2016, Volume 47, Issue 2, pp 291–299