I am an assistant professor in the Computer Science Department at Carnegie Mellon University. I am also a part-time research scientist at Google Magenta.
My research goal is to build music technology that enables a broader set of users to engage with music on a deeper level. To this end, I improve ML methods for generative modeling of music and other multimedia, and build interactive systems which allow anyone to leverage resultant models.
Selected Publications (Full List)
- Melody Transcription via Generative Pre-training In ISMIR, 2022.
- It's Raw! Audio Generation with State-Space Models In ICML (Long Talk; Top 2%), 2022.
- Codified Audio Language Modeling Learns Useful Representations for Music Information Retrieval In ISMIR (Best Paper Runner-up), 2021.
- Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music In ISMIR, 2021.
- Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality In NAACL, 2021.
- Enabling Language Models to Fill in the Blanks In ACL, 2020.
- LakhNES: Improving Multi-instrumental Music Generation with Cross-domain Pre-training In ISMIR, 2019.
- Expediting TTS Synthesis with Adversarial Vocoding In INTERSPEECH (Oral), 2019.
- Piano Genie In ACM IUI, 2019.
- Adversarial Audio Synthesis In ICLR, 2019.
- GANSynth: Adversarial Neural Audio Synthesis In ICLR, 2019.
- The NES Music Database: A Multi-instrumental Dataset with Expressive Performance Attributes In ISMIR, 2018.
- Semantically Decomposing the Latent Spaces of Generative Adversarial Networks In ICLR, 2018.
- Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition In ICASSP (Oral), 2018.
- Dance Dance Convolution In ICML, 2017. * Indicates equal contribution
- (2022-Current) Research Scientist at Google Magenta.
- (Summer 2018) Internship at Google Magenta (Music generation w/ Ian Simon and Sander Dieleman)
- (Summer 2017) Internship at Google (Speech recognition w/ Bo Li and Rohit Prabhavalkar)
- (Summer 2016) Internship at Google Search
- (Summer 2015) Internship at Google Play Music (MIR w/ Nicolas Boulanger-Lewandowski)
- (2011-2014) Mentor for UT Freshman Research Initiative w/ Joel Lehman and Risto Miikkulainen
- (Summers 2011-2014) Internships at UT Applied Research Laboratories, Qualcomm, and two startups
- UploadVR Beat Sage Update Adds 90 Degree Levels, Walls And Single Saber Mode
- UploadVR Get Rhythm: How Beat Sage Uses AI To Create Beat Saber Maps
- Road to VR This 'Beat Saber' Project Uses AI to Generate Custom Beat Maps for Any Song
- UploadVR New AI Tool Turns Any Song Into A Custom Beat Saber Map, And It Really Works
- Stereogum Watch The Flaming Lips Play A Bowl Of Fruit At Google I/O
- Business Insider A Google intern helped build an AI tool inspired by 'Guitar Hero' to let rookies play piano
- The Verge Google’s AI-powered Piano Genie lets anyone improvise perfectly by bashing buttons
- Evening Standard Piano Genie: Google's AI programme is like Guitar Hero for the piano world
- Engadget Google’s Piano Genie lets anyone improvise classical music
- MIT Tech Review Machine-Learning Algorithm Watches DDR, Then Creates Dances of Its Own
- The Verge Scientists have taught a neural network to choreograph Dance Dance Revolution levels
- The Register Yet another job menaced by AI! Uh, wait, it says here... Dance Dance Revolution designers
- Vice This Machine Learned to Choreograph by Watching Dance Dance Revolution
- (2020) Released Beat Sage, a web service for automatically creating Beat Saber levels (link)
- (2020) Download files from Google Drive on the command line (link, code)
- (2019) PhD dissertation on music, AI, and interaction (pdf)
- (2018) Transcribe a batch of solo piano recordings to MIDI (link)
- (2017) PhD qualifying examination (pdf)
- (2016) Master's thesis on convolution-based cross-synthesis, (pdf, code, vst)
- (2015) Prototype for MOOC on computer music fundamentals using Web Audio API (link)
- (2015) Mobile-friendly, networked musical controller (demo)
- (2014) Multichannel convolution reverb plugin (screenshot, code, windows vst)
- (2013) Undergraduate thesis on musical instrument mimicry (pdf, examples, vst)
- (2013) OpenGL 3D spectrogram (page, code)
- (2012-2014) Played keyboard for Food Group
Last updated 2023/8/28