Researchers at UC San Diego are shifting the focus of generative AI from automated song creation to interactive, instrument-like tools. According to Ph.D. student Zachary Novack, the future of music technology lies in systems that artists can “misuse” and shape in real-time, rather than standalone generators that produce finished tracks at the push of a button.
Why is the industry moving away from “one-shot” AI music?
The current market trend of “press-a-button” music generation is being challenged by developers who view it as a novelty rather than a creative tool. Zachary Novack, a researcher at the UC San Diego Jacobs School of Engineering, argues that these systems lack the nuance required for genuine artistic expression. Novack, who is set to join Spotify, notes that while “gigantic companies” focus on massive song generators, the next evolution of the technology involves systems that function like instruments. This shift prioritizes responsiveness, allowing musicians to influence pitch, volume, and timbre during a live performance.
In a recent experimental performance, researchers used a generative model trained on whale sounds to accompany a live cellist. The system acted as a “generative delay,” responding to the performer in unpredictable, creative ways.
How are researchers making AI music more interactive?
Real-time interaction requires significant technical optimization. Novack’s research, alongside faculty advisors Berg-Kirkpatrick and Julian McAuley, focuses on shrinking generative models so they can run locally on hardware. By reducing the computational load, these models can provide the low-latency response times necessary for stage use. This approach contrasts with cloud-based generators that are typically designed for static, high-latency output. The goal is to move the AI from the background of the production process directly into the hands of the performer.
What is the role of artists in AI development?
Successful creative tools depend on involving musicians throughout the development cycle, according to presentations at the 2025 UC San Diego GenAI Summit. Novack emphasizes that artists should not be treated as mere end-users who receive a finished product. Instead, they should act as collaborators who define how a system behaves. This philosophy is gaining traction in academic circles; Anthony Wang, a master’s student at UC San Diego, recently earned second place in the efficiency track at the IEEE International Conference on Multimedia Expo’s Academic Text-to-Music Generation Grand Challenge for his contributions to the field.
| Development Approach | Primary Goal |
|---|---|
| Generative Song Models | Creating finished, “one-shot” audio tracks. |
| Instrument-Style AI | Interactive, real-time creative collaboration. |
Frequently Asked Questions
- Can AI replace live musicians in a performance?
- Researchers like Zachary Novack argue against this, suggesting instead that AI should serve as an extension of the performer, functioning like an instrument rather than a replacement.
- Why is local execution important for music AI?
- Running models locally reduces latency, which is essential for real-time interaction during live music performances where even millisecond delays can disrupt the flow.
- What is the “efficiency track” in AI music competitions?
- It is a category in technical challenges, such as the IEEE International Conference on Multimedia Expo, that rewards models capable of generating high-quality music with minimal computational resources.
Join the Conversation
How do you think AI will change the way musicians perform on stage? Share your thoughts in the comments below, or subscribe to our newsletter for the latest updates on generative technology.
d, without any additional comments or text.
[/gpt3]
