About
Email:
I am an Assistant Professor in the Computer Science Department at Carnegie Mellon University, and also a Research Scientist at Google DeepMind on the Magenta team (part-time).
I am recruiting PhD students for Fall 2025. If you are interested, please see this page for more information about submitting an application.
My research goal is to develop and responsibly deploy generative AI for music and creativity, thereby unlocking and augmenting human creative potential. To this end, my work involves (1) improving machine learning methods for controllable generative modeling for music, audio, and other sequential data, and (2) deploying real-world interactive systems that allow a broader audience—inclusive of non-musicians—to harness generative music AI through intuitive controls.
I am particularly drawn to research ideas with direct real-world applications, and my work often involves building systems for real users to be evaluated in-the-wild. For example, my work on Piano Genie was used in a live performance by The Flaming Lips, and my work on Dance Dance Convolution powers Beat Sage, a live service used by thousands of users a day to create multimodal music game content.
Previously, I was a postdoc at Stanford CS advised by Percy Liang. Before that, I completed a PhD at UCSD co-advised by Miller Puckette and Julian McAuley.
News
- 🎤 (Oct 2024) Invited talk at SANE 2024.
- 📜 (Oct 2024) Launch of Copilot Arena, a VSCode extension for evaluating LLMs for coding assistance.
- 📜 (Oct 2024) Three extended abstracts to appear at ISMIR Late Breaking Demos.
- 📜 (Oct 2024) One paper accepted at the NeurIPS 2024 Audio Imagination Workshop.
- 🌐 (Aug 2024) Official launch of Hookpad Aria, a Copilot for songwriters.
- 📜 (Jun 2024) Our work on Music-aware Virtual Assistants accepted at UIST 2024.
- 📜 (Jun 2024) Two papers accepted to ISMIR 2024.
- 🪑 (Apr 2024) Named as the Dannenberg Assistant Professor of Computer Science.
- 🛎️ (Apr 2024) Serving as Senior Program Committee Co-chair for ISMIR 2024 - record number of submissions (415).
- 🌐 (Mar 2024) A Copilot-like tool for musicians featuring the Anticipatory Music Transformer was launched in beta.
- 📜 (Mar 2024) Music ControlNet to appear in TASLP (IEEE/ACM Transactions on Audio, Speech, and Language Processing).
- 📜 (Mar 2024) Anticipatory Music Transformer to appear in TMLR (Transactions on Machine Learning Research).
- 🌐 (Mar 2024) Launch of MusicFX DJ Mode, a real-time music audio generation tool developed by my team at Google.
- 🗞️ (Nov 2023) SingSong incorporated into Google DeepMind’s Music AI Tools.
- 📜 (Nov 2023) Work presented at HCMIR Workshop by Michael Feffer.
- 📜 (Nov 2023) New preprint on controllable music gen led by Shih-Lun Wu (applying to PhD positions!)
- 🗞️ (Oct 2023) Interviewed for Pitchfork article about MusicLM
- 🎤 (Oct 2023) Invited talk at Stanford HAI Conference (recording, slides)
- 🧑🏫 (Oct 2023) Guest lecture for CMU LLM Course (slides)
- 👋 (Oct 2023) New PhD students: Irmak Bukey and Wayne Chi
- 🧑🏫 (Sep 2023) Started as Assistant Professor at CMU
G-CLef
I lead the Generative Creativity Lab (G-CLef) at CMU. Our mission is to empower and enrich human creativity with generative AI. We focus primarily on the intersection of music and AI, though we also work on other applications such as code, gaming, and language. Please visit this page to learn more about our research interests and to apply.
PhD students
CSD PhD student
CSD PhD student
Coadvised w/ Ameet Talwalkar
Visitors and affiliates
Visiting researcher, KAIST
Visiting researcher
S3D PhD student
Alumni
LTI MS student
Now PhD @ MIT EECS
Music and Technology MS student
Now Researcher @ Michigan CS
Supporters
The G-CLef lab is supported by generous contributions from:
