Ashok Vardhan Makkuva
 |
Postdoctoral researcher |
About
Hi there! I am a postdoctoral researcher at EPFL working with Michael Gastpar, Martin Jaggi, and Caglar Gulcehre. My primary research interests are in interpretability and explainability of LLMs. In particular, I am interested in explaining the fundamental sequential modeling capabilities of various language models such as Transformers and SSMs, and explaining the key differences between these classes of architectures. Some recent works along this theme are: Fundamental limits of prompt compression, Attention with Markov, Local to Global, and Constant depth suffices.
Earlier, I got my PhD in Electrical and Computer Engineering from UIUC working with Pramod Viswanath and Sewoong Oh. During my PhD, I have also had the pleasure of collaborating with Jason Lee and Sreeram Kannan (Founder and CEO of EigenLayer).
Prior to that, I graduated from IIT Bombay with a B. Tech. (Honors) in Electrical Engineering and Minors in Mathematics working with Vivek Borkar.
News
Aug 2024: Excited to share that I will be giving a tutorial at NeurIPS 2024 with my amazing colleagues Bingbin Liu and Jason Lee, titled Sandbox for the Blackbox: How LLMs Learn Structured Data?
Jul 2024: Amazing experience presenting our four papers in the Mechanistic Interpretability and Theoretical Foundations of Foundation Models workshops at ICML 2024: Fundamental limits of prompt compression, Attention with Markov, Local to Global, and Constant depth suffices.
Jun 2024: Our fundamental work on the prompt compression for LLMs is accepted for the Oral track at the Theoretical Foundations of Foundation Models Workshop at ICML 2024.
Mar 2024: Talk on Attention with Markov at ETH Zürich hosted by Thomas Hofmann.
Feb 2024: Our new work on principled analysis of transformers is now available on arXiv: Attention with Markov.
Dec 2023: Talks at UC Berkeley and Stanford on LASER.