Direkt zum Inhalt

First Adventure With Claude And Synfire

Autor andre
Teaser Image Full

After months of implementing the MCP connector I was eager to put it to a first comprehensive test. Will Claude be able to figure out the proper usage of more than 30 diverse tools? Will that lead to anything useful, or will we just get another variation of trivial elevator music? I will post my progress on this experiment here. Keep coming back as I add more results and insights. A video is found a few posts down.

First off, I must share my briefing, so you get an idea what a briefing for the new connector looks like (you will need to write your own). It's a markdown file, among others that come with Synfire in order to explain its paradigm and workflows to the MCP client (the AI you connect to it). 

Note that your briefing is permanent – it becomes part of the system prompt the LLM will read everytime you start a new chat. You can also write one or more skill.md files to teach the AI how to accomplish things. What exactly works best in these files will be an ongoing community effort that I hope we can make some progress on after the launch.

Note: This feature comes with the next update

Here's the original file:

Briefing

After you changed parts of an arrangement, please play them to me so I can follow your progress.

Instruments & Sounds

  • Search my rack module library for presets that match the requested style. I have orchestral libraries for VSL, Spitfire SSO, Tides (Rachel Portman), Orchestral Tools "Ark 2", and more.

  • Use articulations when they are available. Orchestral sounds (VSL, Spitfire SSO, Orchestral Tools) all support them.

  • For electronic, ambient, and dance music, I have saved named states inside rack module presets that you can use.

  • Avoid using the bland "Soundcase" plug-in for anything but drums.

Structure

  • Build a container structure with sections and sub-sections (e.g. song parts) that reflect the narrative of the piece. The more detailed the structure is, the more comfortably you and I can navigate it for surgical editing.

  • Make use of aliases for reprises and recurring parts.

  • Compose build-ups and drops (drops may be as short as 1-2 bars).

  • Avoid mechanical predictability – after an introduction has settled the theme, use varying durations for subsequent reoccurrences and new sections (shorter means more surprises).

  • Use colors to distinguish major sections of music.

Harmony

  • If you can create interesting chord progressions from your knowledge, that's great. I also have extensive phrase libraries with harmony progressions you can use. Also use the factory "Stray Chords" creatively.

  • Avoid the predictable one-chord-per-measure rhythm. Sprinkle in faster chord changes, or hold a single complex chord for longer to build suspense.

  • Use chromatic planing in the orchestral genre towards climaxes (assign a chromatic parameter to the defaultinstrument).

Figure

  • Build figures with parameter_edit by concatenating multiple short fragments (1m) that repeat and evolve.

  • Use the parameter_edit tool to transform partial spans of a figure (or any parameter) in order to create variations and motif development. This way you can stitch together fast moving ostinatos without editing individual segments and symbols.

  • Make use of parameter aliases to establish a motif that reoccurs in multiple places and that you can later edit freely for experimentation.

  • I have libraries with names like "Construct" that include building blocks for piano and orchestra.

Other Parameters

  • The modulation and expression parameters often control the dynamics of orchestral instruments. Also check if there are useful custom controllers specific to a device or sound.

  • You can transfer parameters and phrases from my other projects. Ask me for project names that may provide useful raw material.

Comments

Mi., 18.11.2026 - 22:29 Permalink

My first prompt was this:

Please compose and arrange a nocturnal for piano and chamber orchestra. Use the "NI Noire" piano and my VSL orchestral rack module presets.

Build a narrative structure with multiple parts and sub-parts, and use aliases for reprises and recurring motifs. Populate the structure with phrases for each instrument. Start slow and ambient-like, impressionistic, and develop a rising density up to a climax that ends with a drop and continues with recurring variations.

Build chord progressions in D minor that suit the dark theme (key changes are welcome). Look into my phrase libraries, where you will find construction sets for orchestra and piano. Also consider harmony Factories (e.g. "Slow Counter Moves") to generate new cinematic chord progressions.

Make use of articulations and the pause parameter. Use the parameter_edit tool to create variations and motif development. Use the modulation parameter to let the instruments "breathe" and waves and swells. Add child containers in the denser parts of the arrangement, where you can place faster paced elements towards the climax.

Ask me any time if you are unsure which direction to take. We want to make this a great cinematic listening experience.

Claude worked for about 20 minutes and came up with this:

(original audio attached below)

Claude's own words:

The nocturne is written and playing from the top in Synfire now: 94 bars, about six minutes, saved as Arrangements/Nocturne in D minor.cognac. I have only auditioned the Theme and the Brink/Summit climax myself, so the full run-through you are hearing is the first complete listen.

Structure

Part Sub-parts What happens
I. Nightfall (16 bars) Mist, Stirring Felt piano drops over tremolo violas and low strings. The theme's rising fourth assembles note by note, then clarinet and English horn echo the motif.
II. Nocturne Theme (16) Theme A, Theme A' Noire piano states the theme over a nocturne arpeggio with pedal, strings pad and pizzicato basses. A' sequences it upward and turns toward F minor.
III. Drift (12) Undertow, Counter Moves First 8 bars of harmony come from the Stray Chords factory, starting from your "Slow Counter Moves" settings. Strings enter staggered; a distant horn plays the motif.
IV. Ascent (14) Gathering, Surge, Brink Rising bass line, motif climbing in sequence, tempo 64 → 72. Brink is two bars of string tremolo and a piano run.
V. Climax (8) Summit, Drop Theme in octaves on violins, flute, clarinet and horn over 16th-note violas and full piano. Then a two-bar drop to felt piano and tremolo violas at 52 bpm.
VI. Afterglow (28) Reprise, Variation I, Variation II, Mist Returns, Last Breath Reprise, then the theme in G minor against its own inversion in the cellos, then its retrograde on felt piano with an augmented English horn.

How your requests were handled

  • Aliases: Reprise and Mist Returns are container aliases. The theme lives in one place (piano RH in Theme A), and every motif echo and the climax tune are parameter aliases of it, so editing that one figure changes them all. The left-hand arpeggio and pedal work the same way.
  • Variations: Theme A', the Gathering sequence and both Afterglow variations are copies developed with parameter_edit (transpose, upside-down, reverse, stretch).
  • Child containers: four motif echoes, plus Pulse (staccato strings), Runs (piano 16ths) and Flurries (woodwinds) in the Ascent.
  • Pause and modulation: pause staggers entries and empties the Drop. Every string and wind instrument has modulation swells, with a long ramp through the Ascent.
  • Articulations: legato, tremolo, staccato, pizzicato and marcato on the VSL instruments.
  • Libraries: Construct Debussy piano phrases in Drift; Construct Orch flute, clarinet, string wave, legato line and clarinet staccato phrases in Drift and Surge.

Things you should know

  • Intro piano is my own writing. The Construct Piano "Sparse Phrasing" phrase rendered too busy and low for the opening, so I replaced it.
  • Two piano instruments share the Noire Pure channel (RH and LH) so each hand can have its own figure; a third uses Noire Felt for the intro, Drop and ending.
  • Module gains are down 4 dB (brass 5). The climax peaked at +3 dB before that; I have not re-measured since.
  • No breath before the Summit. The winds hold straight through the Brink into the climax.

Tell me which sections you want reworked after listening — the climax balance and the Drift harmony are the two I would check first.

nocturne-in-d-minor-a.mp3

Mi., 18.11.2026 - 22:48 Permalink

Impressive how it handles Synfire, but this is a raw diletantic first sketch at best (flat strings, those hammering chords in the climax, no discernable theme, I could go on ...). 

But I see how this could be a nice skeleton for me to work on, so I'm not disappointed yet.

Important to note:

  1. Apart from a sparse piano part in the intro, Claude didn't "write" the music. It searched and used my libraries (most are also available for download on this site). What it actually did – and that surprised me – it built a concept, a structure, and harmonies based on the millions of talk/discussions/books etc. it was trained on. Synfire is the hands, eyes, and ears it uses to put these (abstract) ideas into action.
  2. This was run with Fable, but earlier experiments showed that much less capable models (e.g. Sonnet) are doing fine as well.
  3. The primary purpose of the MCP connector is not to compose a finished product. Its job is to prepare something for you to work on, so you don't have to start from zero.
  4. Claude "listened" to individual instruments and the mix to make adjustments (it can ask Synfire for audio spectrograms).
  5. It's totally comfortable with using articulations and custom controllers. In another session I tested and asked "How would this piece sound with the Rachel Portman Tides library?" and it translated and reassigned articulations and controllers in a couple minutes, so I could listen to (too bad I didn't record this – will show a rerun later on).
  6. Claude didn't use color, used a boring 1-chord-per-bar harmonic rhythm, and the parts are way too long. That was my fault. I updated my briefing accordingly (the copy you read above).
  7. As long as you can describe what you want, current AI can somehow handle that. If you giveit generic instructions like "write somethign cool" you'll get boring stereotypes and elevator music.

Mi., 18.11.2026 - 22:52 Permalink

Here's my next prompt:

The structure/narrative is beautiful. But overall it's too predictable and unsurprising: The pace of 1 chord = 1 measure (mostly) and little variation in the rhythm is tiring. The strings could use a bit (albeit slow) movement – simple 1/2 note steps are fine, nothing fancy. Harmony could be more daring, possibly chromatic in the climax (use chromatic parameter for default instrument to achieve chromatic planing)

The next iteration was already much better (almost same picture, only the parameters changed). Claude's summary:

The revision is saved and playing from the top. I checked the rendered notes for the theme, the Brink and the Summit, but have not listened to this version, so the balance is unverified.

What changed

  • Harmonic rhythm: chords no longer change once per bar. Most sections now mix 1½-bar, one-bar and half-bar chords, so the changes land in unexpected places.
  • More daring harmony:
    • The theme now sits on a descending lament bass (D–C–B–B♭) with secondary dominants.
    • Cadences use tritone substitutes (E♭7, B♭7, G♭7, A♭7) and Neapolitan chords.
    • Theme A' has a chromatic slide (Gm7 → G♭maj7).
    • Gathering climbs on a chromatic bass with passing diminished chords.
  • Chromatic planing in the climax, using chromatic on the default instrument:
    • Brink: the whole orchestra steps up by semitones per half bar (A7sus → B♭7sus → B7♭9), then falls back to A7♭9.
    • Summit: the theme and everything under it rise a semitone per bar for three bars (+1, +2, +3), then drop back onto the dominant before the Drop.
  • Strings move: the pads are now slow half-note steps between chord tones. Each section loops at a different length (violins 2 over three bars, the others over two), so the voices phase against each other instead of re-bowing together every bar.
  • Rhythm:
    • The piano left hand alternates a bar of eighths with a bar that holds and then runs in triplets.
    • The staccato strings in the Ascent play a 3+3+2 pattern instead of straight eighths.
    • The pizzicato basses get an off-beat pickup.
    • The Theme part has a rubato tempo curve with ritardandi at the phrase ends.

Things to listen for

  • Suspensions: where a chord changes mid-bar under a held piano bass note, you get a brief suspension. I left these in as colour, but say so if any sound wrong rather than dark.
  • Reprise: it is an alias of Theme A, so it picked up the new harmony automatically but has no rubato of its own.
  • Summit top note: the planing pushes violins 1 up to a high B♭ at the peak. If that is too shrill, I can cap the rise at +2.

Do., 19.11.2026 - 10:56 Permalink

Claude is 20 minutes thinking ..how many tokens is that ?
You need a API key for this. 
 

Do., 19.11.2026 - 11:11 Permalink

This was a demo of Fable that I wanted to try before the promotion expired. You don't need such a high-end model for this. I get almost the same results with Opus, and even Sonnet does a great job. I'm doing all this on a $20 subscription. The normal ChatGPT subscription did fine as well.

Do., 19.11.2026 - 11:21 Permalink

It is still not clear to me how this is going to work with ChatGPT.

The Codex quota gets used up very quickly.

The regular ChatGPT plus subscription seems to have almost unlimited usage and works great, but it cannot be used for composing with Synfire because it does not have access to Synfire.

Do., 19.11.2026 - 11:59 Permalink

As long as your chatgpt subscription can talk to mcp servers there are ways of getting it to talk to a local synfire mcp server. You'd need to create secure tunnels or otherwise proxy the mcp server so it is exposed on the internet, or better yet configure local agents to bridge the gap. I'm sure chatgpt can help solve your problem with chatgpt, if not ask a different ai model/service for the instructions on how to fix your chatgpt problem. Then again if you havent paid for mcp support in your subscription, that will be the first step in achieving what you want.

Do., 19.11.2026 - 12:00 Permalink

Any frontier or local AI will have access to Synfire with the next update – that's what Synfire's MCP connector is built for.

For motivation, I will cut to the chase. This is the result after my own substantial edits:

I might somehow have managed to do something similar without the help of Claude (because Synfire is already a great assistant), but I would never – ever – get this far in so little time. This overhaul took me 2-3 hours total with a few rests in between.

To get this result, I had to substantially edit Claude's draft:

  1. A language model can think up a concept, build a structure, and populate it with phrases that it "thinks" might fit. This is very helpful, but none of its choices were great. I had to replace most of them.
  2. Harmony was surprisingly fine, I kept most of it. The chromatic planing in the Ascent and Climax is ... wow!
  3. The sections were way too long.
  4. There was no discernable theme. The melody was cheesy and trivial. I replaced it by two simple intervals that reoccur throughout the piece as a leitmotif of sorts.

Do., 19.11.2026 - 12:13 Permalink

I used Synfire with ChatGPT without an active subscription. The local server setup is a bit hard to find, but possible.

Do., 19.11.2026 - 12:19 Permalink

"Yes, that is also possible with the standard ChatGPT, but with the Plus subscription, ChatGPT is much more powerful."

Do., 19.11.2026 - 12:30 Permalink

"I'm sure chatgpt can help solve your problem with chatgpt, "
i think it is no problem to figure this out with ai.

Do., 19.11.2026 - 13:41 Permalink

I am curious about how this will work for the average Synfire user, since they will probably not work in the same way as André, who has extensive knowledge of Synfire.

Would it be an idea to start with a MIDI composition and use that composition as the basis for the project?

Or, in other words, to build the composition around an existing MIDI file first?

Do., 19.11.2026 - 13:55 Permalink

What I Would Like to Add to the MCP Server

Four simple commands:

  • AddMIDINotes — Insert multiple MIDI notes at once.
  • EditMIDINotes — Modify pitch, rhythm, and velocity.
  • DeleteMIDINotes — Delete selected MIDI notes.
  • ReadMIDINotes — Read existing MIDI notes so that the AI can further develop a composition.

For example, you could simply say:

Compose 16 bars for a string quartet in D minor, with a contrapuntal second violin part. Use a slow tempo of 72 BPM. Then make the viola part more rhythmically interesting.

I could then generate the individual MIDI notes and send them directly to Synfire via MCP, provided André makes these functions available.

My proposal is to avoid the complex Figure system for this purpose and instead use a direct interface for manipulating MIDI notes in Synfire.

One important question: Has André already made the MCP server available for users to install, or have you only seen his demonstration on the Cognitone forum? This determines whether we can already establish this direct MIDI connection.

Do., 19.11.2026 - 13:57 Permalink

Can Both Composition Methods Work Side by Side on the Same MCP Server: Direct MIDI Note Composition and Figure-Based Composition?

Yes, absolutely. A single MCP server can support both direct MIDI note composition and Synfire's existing Figure-based composition method.

In fact, I would recommend keeping both methods available side by side, as each offers different advantages.

Two Composition Methods on One MCP Server

Feature MIDI Composition Figure-Based Composition
Approach Absolute MIDI notes Symbolic musical patterns
Pitch Explicitly defined Dependent on harmony
Rhythm Direct note positions Through Figure symbols
AI Complexity Relatively simple More complex
Automatic Harmonization Not automatically Yes, through Synfire
Best Suited For Precise notation, counterpoint, orchestration Harmonic variations and arrangements

The MCP server could, for example, provide two separate groups of commands:

MIDI Tools

  • AddMIDINotes
  • EditMIDINotes
  • DeleteMIDINotes
  • ReadMIDINotes

Figure Tools

  • AddFigures
  • EditFigures
  • DeleteFigures
  • ReadFigures

The AI could then automatically select the appropriate method depending on the composition task.

Important Consideration for André

Synfire already includes a Pitch (P) symbol type for absolute pitches.

This means that direct MIDI note composition might be implemented using existing Figure structures, without André having to develop an entirely new MIDI engine.

Synfire also allows different symbol types in separate segments, although not within the same segment.

For example, within a single arrangement, the AI could compose the violin melody using absolute MIDI notes, while generating the piano accompaniment using harmony-dependent Figures.

The existing MCP server is still experimental. André would need to confirm whether direct MIDI note editing is already supported through its available commands.

My Recommendation

I would suggest asking André to make both composition methods available through the same MCP server, with an explicit choice between:

  • MIDI Mode: Direct composition and editing of individual MIDI notes.
  • Figure Mode: Composition using Synfire's symbolic Figures and harmony-dependent musical structures.

Ideally, both methods could even be combined within a single musical composition.

This would allow the AI to work directly with MIDI notes whenever precise control over pitch, timing, duration, and velocity is required, while retaining the powerful harmonic and transformational capabilities of Synfire's Figure system.

The main advantage is that users would not have to switch between different AI systems or MCP servers. Both composition methods could coexist and complement each other within the same Synfire environment.

Do., 19.11.2026 - 15:46 Permalink

Thanks for your suggestions. The server already supports almost everything you can do with Synfire except Snippets. It can edit all parameters in detail. So yes, if you want, you can ask your AI to compose in absolute pitch symbols (and harmonize that or whatever you want it to do).

Oh, and the server is not experimental anymore. You will be able to use it soon.

Do., 19.11.2026 - 15:56 Permalink

The connector is especially useful for beginners. You can ask it to do anything for you:

  • Projects: create, open, save and close arrangements, racks and libraries.
  • Structure: build and reorganize an arrangement’s containers (sections), including aliases, length, time signature, tempo changes, colors, and naming.
  • Instruments and sounds: add instruments and assign sounds. Search by name, category or articulation across your device descriptions.
  • Rack and plug-ins: set up rack modules with your installed AU/VST/VST3 instruments and effects. Load presets, recall saved plug-in states, and control volume, send, solo and mute.
  • Device descriptions: read, create and edit device descriptions for plug-ins and hardware, including sound banks, articulations and custom controllers.
  • Parameters: read, write and edit musical parameters such as harmony, rhythm, figure, dynamics, flow, velocity, take, tempo and expression. Use templates, copy, transpose, reverse and stretch.
  • Libraries: search phrase libraries by keyword or by similarity to an example. Drop phrases into your arrangement, or save your own.
  • Factories: generate new phrases with Synfire’s algorithmic Factories. Tune their settings and create variations of phrases you like.
  • Playback and listening: control the transport and preview parameters or note sequences. Read back the rendered MIDI output, or get a spectrogram so Claude can judge the sound.
  • Navigation and notes: jump to the relevant page or view in Synfire. Claude keeps notes per project and globally, so it can pick up where it left off.
  • Guide: a built-in guide covers workflows and the user manual.

Do., 19.11.2026 - 20:56 Permalink

@janamdo said: 

My proposal is to avoid the complex Figure system for this purpose and instead use a direct interface for manipulating MIDI notes in Synfire.

Interestingly, the figure system is what Claude makes Claude thrive in the first place. The AI has immediately grasped the different symbol types and is using them to full advantage. Rather than spending thousands of tokens on reasoning about absolute harmony and all its intricacies, it just thinks up the chords and relative movements. Humans may sometimes struggle with functional expressions but an AI loves them!

Fr., 20.11.2026 - 15:32 Permalink

Very impressive demo video!   Thanks for the research, the work, and the video!

I look forward to future support of all Snippets operations.  I think the recipe I like to use with Snippets is a good fit for being driven this way.

 

 

Sa., 21.11.2026 - 23:01 Permalink

Indeed impressive, but not mind blowing like i had with other composer like Nicolas Shuelte  from Switserland.
I had now time to see it  full ,  but how is the working with Vienna ensemble Pro server and Synfire possible ?
I want to approach first the simpler midi notes approach composing.