Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion

To demonstrate how our system can be used effectively as a compositional tool, we provide three example videos. A musician specifies musical characteristics characteristics in terms of the rules (e.g. fairly sparse excerpt, following a simple I-V chord progression in C major, etc), and the system generates 3-5 samples per request. The musician selects a favorite sample, loops it as accompaniment on a Disklavier piano, and improvises over it. The following three videos feature the musician's performance with the system-generated music.

In Demo 1, the model generated material that suggested a melody. Since the musician wanted to play the second track in the upper register, they first allowed the excerpt to play in full, as generated, and then removed the upper notes from the accompaniment to give room for themselves to play overtop. They chose to use the model’s generated melody as a motif, and further improvise based on it.

In Demo 2, the model generates an excerpt with a steady accompanying triplet ostinato behind a slower-moving descending melody in C minor (that suggests a progression that moves between the I and the V).

In Demo 3, the model generates a sample with a changing note density and texture, and a slightly ambiguous harmonic quality that allowed flexibility in the improvising over it.