Ai composing without MCP

You mean it does the mouse clicks and all to control Synfire? The arrangement you uploaded includes all static figures (MIDI notes). I can hardly believe ChatGPT could have drawn these with the pencil tool, without it knowing how the symbols map to pitch.

I would be more convinced if it had used containers, dynamic figures, and the harmony parameter. That would have resulted in only a few short figures and the rest would unfold automatically.

Also, for these very simple forms you don't need AI. Sketching an arpeggio and drawing/recording a few lines gets the job done. AI would be more interesting for developments, unusual harmony, dense textures, long build-ups. Things that are more fiddly to do by hand.

We already have a MCP interface to Synfire (lab version). Would be interesting to check if it can "write" meaningful Figures and Harmony parameters. That would be fun. But don't expect any cool music to emerge. A LLM has no judgment and taste (it can't even hear the music).

The whole point of astra llm is that it can operate without an agent, although personally I think you're mad giving it that level of access. Although if Synfire release their agent access to synfire then you can use any LLM that is compatible without giving it access to your whole machine.
Mind you if all you want is a bunch of elevator music, it might be more economical to post a request on fiver for someone to compose some tunes for you.

I think it's definitely interesting to see what Claude or Astra would do if they had MCP access to Synfire's containers, parameters, rendered output, etc. That's probably a one day project, most of the work sitting in the Synfire to JSON converter.

However the problem is not control of the software. It's lack of musical knowledge. LLMs can talk about music quite well (if you want to learn more about music theory, it's always worth a chat). What they can't do is look at a piano roll (rendered output) and tell whether those bars match the user's ideas or not.

I would be very surprised if that MCP experiment would be able to generate some non-trivial and authentic music. Probably it's just because people don't talk about this a lot in detail online (no training material). I strongly believe that anything beyond a mere imitation of stereotypes is still a human domain.

You also noticed that even simple elevator music already consumes tons of compute, water and energy. Humans are so much more effecient. Cognitone's philosophy is to help humans explore and build on their own ideas - not to replace them.

We shold probably change the subject to "AI composing with MCP" now. I just did a very quick & dirty test (no time to dive into this right now). I added a rudimentary parameter creation tool to the MCP we already have and gave it to Claude Opus.

Yes, it's real fun to type instructions and see parameters appearing in Synfire like magic. But no, its super slow, chatty, and the results struggle to meet even the bar of evelator music.

Claude spent 8 minutes burning tokens, electricity, and water on a simple "cinematic piano intro", which is essentially a continuous arpeggio (green) and a (blue) melody you could sketch or record in 30 seconds. It spent a lot of thought on smooth transitions (dynamics, tempo change). The elaboration and thought process was superb (the fun part of this excercise). It's a language model after all. The result was a trivial ramp and not smooth at all.

I'm glad I invested an afternoon to test this now, as it is now pretty clear that, whatever a MCP might eventually help with, composing phrases it is not.

If you can play the keyboard at least somewhat, you'll do a better job than Claude hands down. Even without that skill you get 1000% better results by importing some classical piano music, harvesting it for phrases you like, slowing them down and thinning them out, and some drag & drop action.

An LLM can only know about music as much as humans have talked about it online or in books. How that talk maps to the intricate details and processes involved in writing music, is just not present in the training data.

Here's a second try. Although it's amazing how Claude immediately understands the paradigms of Synfire (we should copy the MCP instructions to our user manual ;-), human beginners are much better by orders of magnitude. 

The conversation required compression half way through (i.e. it exhausted the 1 million token context window!). Imagine the token orgy if Synfire had all its features hooked up to MCP. You'd need a $200/mo subscription to get something done.

Prompt: I have a blank new arrangement open for you to use. Create a smooth cinematic piano theme with multiple parts, slow development and variation, and interesting dark harmony.

Composing a cinematic piano theme in Synfire.Composing a cinematic piano theme in Synfire.

I'll help you build a cinematic piano theme in Synfire. Let me first load the Synfire tools and understand the current state of your arrangement.Synfire-Integration verwendet, geladene Tools

I've got the full picture: a blank 32-bar arrangement, a General MIDI soundset in the global rack (grand pianos, slow strings, warm pads), and you're currently in the Harmony editor.

Here's my plan before I start writing: a dark D minor cinematic theme at ~66 BPM, built as five nested sections that develop slowly — Intro → Theme → Variation → Climax → Outro — across three parts (piano right hand for the melody, piano left hand for flowing broken-chord accompaniment, and a soft sustained strings/pad layer for cinematic depth). The "dark" color comes from a lament progression (i–♭VI–iv), a Neapolitan ♭II (E♭maj7), and a Phrygian descent in the climax.

Let me first set the tempo, which will also confirm how instruments get created here.Synfire-Integration verwendet, geladene Tools

Everything's in place. The composition is fully written across all five sections and three parts, and I've confirmed it's stored correctly.

Here's what I built — a dark cinematic piano theme in D minor at 66 BPM, structured as a slow arc:

Intro (4 bars) — a distant, sparse entrance. Soft open fifths in the pad, a fragment of melody drifting in over Dm → B♭maj7 → A7.

Theme (8 bars) — the main lyrical idea: a descending piano line over a lament-flavored progression (Dm → B♭maj7 → Gm7 → A7), with the melody leaning on the C♯ leading tone over the dominant and touching the Neapolitan color of E♭maj7.

Variation (8 bars) — the same bones developed higher and more ornamented, eighth-note motion in the left hand, more harmonic restlessness (Dm7, E♭maj7, Asus4 → A7♭9).

Climax (8 bars) — octave-doubled melody at the peak, bold two-octave arpeggios underneath, full sustained strings, then a tapering descent.

Outro (4 bars) — everything thins out and fades, ending open and unresolved on the fifth rather than a clean home chord.

The darkness comes from a few deliberate choices: the descending lament progression, the Neapolitan E♭maj7 borrowed into the key, Phrygian color in the melody, and A7♭9 dominants that never quite let the tension settle.

One thing you need to do manually (Synfire leaves sound and range assignment to you): in the rack, assign Yamaha Grand Piano to both inst1 (right hand / melody) and inst2 (left hand / accompaniment), and Slow Strings or Warm Pad to inst3. Then set pitch ranges so the octaves land right — melody high (~C4–C6), left hand low (~C2–C4), pad in the middle. The melody is written in relative notation, so it's currently folding into a low register until you give it a proper range; once you do, the contour will open up as intended.

Once ranges are set, I can re-check the rendered notes and fine-tune anything that clashes — the melody over the E♭maj7 bars is the spot most worth a second look. Happy to also adjust tempo, dynamics, or section lengths if you want it darker, slower, or longer.

Here's the result (mp3 attached).

I mean it looks nice. Claude understands how Figure works and created 5 parts. There is visible variation and development. Synfire cares about harmony, so it's easy for the AI to avoid "ouch" mistakes. Rhythm however is super static and borders on annoying. With rhythm, it obviously it helps to have a body ;-)

The killer argument however: You can do this easily yourself and - if you are a beginner - learn a skill.
 

One thing surprised me though. Claude seems to be quite good at generating chord progressions (not the one above). If you give it enough hints. There's obviously a lot of that in its training data and if it can talk about harmony, it can also tell you the chords.

No MCP needed: Copy/paste or just typing works fine.

Couldn't resist and asked it to improve the boring rhythm.

I've changed the sounds. Nice work so far. Only rhythm is really static and boring. We have a Shift parameter for rubato (which you can't yet access), but beyond that humanization, rhythm should be something more varied than straight quarter notes. How a bout some triplets or quintuplets here or there? Or a few early or late note onsets?

Working out fraction notation for triplet and quintuplet timing.

Good — glad the sounds are in. Let me first confirm how Synfire wants tuplet and displaced-onset notation before I rewrite anything, since I want early/late onsets and triplets to actually render rather than get quantized. Let me check what I already learned in the session.Hat einen Befehl ausgeführt

I didn't test tuplet notation in the earlier session — only straight divisions (1/8, 1/4, etc.). Let me check the ontology doc for how Synfire notates triplets and off-grid onsets.3 Befehle ausgeführ

Rather than guess at notation, I'll test it directly. Let me read the current melody figure in the Theme so I can rework its rhythm while keeping the pitch content.Synfire-Integration verwendet, geladene Tools

Tuplet and off-grid notation both work — the triplets (1/12), quintuplets (1/20), and the syncopated onset "3/4+1/16" all stored intact (Synfire even normalized 3/20 to the dotted form 1/10*). Let me confirm it renders to actual MIDI.Geladene Tools, synfire-Integration verwendet

Confirmed — it renders to clean MIDI with the triplets, the quintuplet, and the syncopated onset all placed correctly, and now in a proper register since you set the ranges. So Synfire notation is: eighth-triplet = 1/12, quarter-triplet = 1/6, quintuplet = 1/10 or 1/20, and off-grid onsets via +1/16 / +1/32. Now I'll rework the rhythm with that vocabulary. Starting with the melody (inst1), section by section.Synfire-Integration verwendet

Compound positions normalize correctly (the quarter-triplet lands right on 5/6, the late onset at 3/4+1/32). Now the Theme melody — syncopations, an eighth-triplet descent, a quarter-triplet turn, the C♯ leading-tone entrance pushed late, and a quintuplet flourish over the final A7♭9.Synfire-Integration verwendet

inst1 is fully reworked. Now the left hand (inst2), which was the most mechanical part — I'll break the straight broken-chords into a mix of rolling eighth-triplet arpeggios, syncopated patterns, and late-onset bass notes, varying the feel bar to bar. Starting with the Theme.Synfire-Integration verwendet

Verified — both parts render exactly as written, harmony fully intact. The rhythm is now varied throughout instead of straight quarters.

Here's what changed across all five sections:

Melody (right hand) now moves through eighth-note triplet runs, quarter-note triplet turns, and two quintuplet flourishes — one cascading over each A7♭9 cadence (Theme and Variation) and a fast descending quintuplet at the climax apex. The leading-tone entrances (the C♯ over A7) now come in slightly late, a written-in rubato lean, and phrases mix dotted, held, and syncopated values rather than even notes.

Left hand was the most mechanical part, so it got the biggest overhaul: the straight broken chords are now a rolling mix of continuous eighth-triplet arpeggios, syncopated off-beat patterns, and bass notes nudged a 32nd late. It builds from a two-note roll in the Intro to driving two-octave triplet arpeggios in the Climax, capped with a double-quintuplet run and accented octave stabs at the peak, then thins back out for the Outro.

A few notes:

I confirmed Synfire's tuplet notation by testing it live — eighth-triplet is 1/12, quarter-triplet 1/6, quintuplet 1/10/1/20, and off-grid onsets write as +1/16 or +1/32. All of it renders to real MIDI, so the variety is baked into the figures themselves, independent of the Shift/rubato parameter. When you do apply Shift, it'll layer micro-timing humanization on top of these structural rhythms.

I deliberately left the strings/pad (inst3) sustained — rhythmic chatter there would fight the melody and muddy the cinematic wash. If you want it to pulse or swell in time, I can add that.

Have a listen and tell me if it's the right amount — I can push further (more aggressive syncopation, swung feel, heavier tuplet density) or pull any section back toward simpler if it's too busy against the harmony.

_______________________________________________________________

Cool insights. And of course I enjoy how Claude loves Synfire's MCP tools. And again, yes it's magic to watch the Figures transform on screen (keep in mind that it gets to see wild nested JSON structures only). 

But this AI has no feeling for rhythm beyond clever talk.

Adding a feature just for quick experimentation is a risky bet. We need to make sure it is really useful first. That will take some more time.

Have to say andre, I'm quite impressed with the tokens that claude generated, it really makes you believe it knows it's stuff. Equally impressive is it working through your mcp to control synfire, even if you havent exposed all of synfire's parameters yet.

What is less impressive is the quality of the actual music it produced and the cost of producing it.

I presume, no amount of work on the MCP will improve that, as the fault lies with the ai model, even if it gets access to additional parameters?

I think the ai models trained on audio do a much better job at producing bland generic music, probably because there is so much audio out there to train against.

I do think as you intermated, using an ai chat tool to discuss musical ideas such as harmony, tempo, rhythm, etc could be useful and much cheaper. It would require the composer to copy those bits and ideas they like and paste/transfer into synfire, but using MCP agents is a recipe for large bills or at the very least, rapid consumption of token credits.

Although an interesting experiment, personally I think your time is better spent elsewhere.

Yes, time is tight. This was still a necessary experiment because AI evolves so fast, we need to learn what it's good for early on. As a last test, I'll check out how it responds to "latin dance music" when I also give it a drum track and bass ... but won't hold my breath.

I see potential for MCP and AI models in scripting  & search though. If we find a useful embedding (vectorization) for figures and parameters, we can have very fast similarity search in libraries, which is a big deal. Imagine you play something and Synfire opens a library with smilar material and selects a suitable phrase pool.

Or MCP control of Factories.

If this feature is separate from Synfire's standard functionality, then users can enable or disable it, just as they have with previous features in Synfire.

The feature needs to be at least somewhat complete to make sense - not the quick half-baked lab setup we have now. Such an investment needs justification.

The bossa nova was promising ... until I got this

Here's what it came up with for a fullly exhausted token budget:

Cool: Claude used the alias feature for the second verse and inserted child containers for breaks (which I hinted at). The harmony is nice. It feels like a song ..... from an arranger keyboard's automatic accompaniment  30 years ago ;-)