Posted
As I'm currently experimenting with an MCP server for Synfire (a prototype is already running), I have to provide the client models with instructions about the concepts and terms of Synfire. Of course, the AI can't pull in the entire online manual only to understand what the user refers to by the term "device description" (actually it can, but that will cost you tokens and context window).
Fun thing is how this exercise is so much easier than explaining Synfire to humans ;-) I can keep it extremely concise, almost like a formal definition. The less verbose, the better.
Here's an example:
Instructions
This server controls Synfire, a desktop application for music composition. Its paradigm is to denote pitch relative to current chords and scales, rather than using static MIDI notes. This makes musical fragments reusable and portable across projects. The philosophy of Synfire is to compose music based on higher-level musical concepts like harmony, form, structure. These can be altered to recompose a piece as a whole without editing a single note by hand. Synfire arranges sections of music on the timeline in a hierarchical structure, rather than on linear tracks. Tracks are simply the result of rolling out the hierarchy for each instrument.
Ontology & Terms
- Arrangement: The primary document that contains a piece of music.
- Instrument: An arrangement may render music for one or more instruments.
- Parameter: A series of numbers, settings, or objects mapped along the timeline (similar to automation lanes). There are many types of parameters that control the rendering of music in different ways.
- Figure: A parameter similar to a piano roll, the notes of which however are relative to current harmony rather than static.
- Symbol: The relative notes in a Figure parameter. Symbols come in multiple types. Each type uses a different reference point in harmony.
- Segment: One or more symbols grouped together in order to form a coherent melodic fragment. Chords can be denoted as vertically stacked symbols.
- Anchor: The symbol in a segment that anchors it on the timeline and defines its reference point in the relation to harmony.
- Harmony: The parameter that provides the chord progression which all instruments must follow. It defines chord, bass, scales, and key.
- Vertical Scale: The scale that moves and changes with every chord. It always starts at the root note of the chord.
- Horizontal Scale: The scale that starts at the root of the key and usually continues across multiple chords.
- Palette: A table of chords and scales the user can pick from to build a harmony progression.
- Phrase: A bundle of one or more parameters.
- Container: A region on the timeline that contains phrases for one or more instruments (similar to folders in Logic Pro).
- Clip: A phrase placed in a container, which limits its duration.
- Structure: A tree of nested containers with the root container at the top, which anchors the piece of music.
- Track: The series of clips on the timeline that results from rolling out the structure for each instrument.
- Library: A secondary document that contains many phrases, parameters, snippets, containers. It serves as a repository for ideas.
- Embedded Library: A library owned by and stored along with an arrangement. It serves as a clipboard for elements not currently used.
- Factory: A generative algorithm that creates parameters and phrases based on musical rules and user settings.
- Snippets: A grid of containers that can be played live, similar to clip launchers of Ableton Live.
- Sketch: A small arrangement to play live, following the user's selections in palettes or progressions.
Sounds & Audio
- Plug-in: Virtual instruments and effects (VST3, AudioUnits, AAX).
- Device Description: A file that contains meta-data about a plug-in or hardware instrument (controllers, sounds, articulations). When a user says "device" they often refer to a device description.
- Sound: A patch or preset provided by a hardware synthesizer or virtual instrument. Device descriptions provide pitch range, category, controllers, and articulations for each sound.
- Rack Module: An audio channel strip that hosts plug-ins (instrument plus effects). It has a device description attached, so Synfire can use a sound's category, pitch ranges, controllers, and articulations.
- Rack: Multiple rack modules that constitute a basic mixing console.
- Global Rack: This rack is always online and all currently open arrangements may use its sounds.
- Arrangement Rack: An arrangement's private rack, the sounds of which are available only to itself.
- Global Instruments: Six permanently available instruments based on sounds of the global rack (piano, chords, bass, guitar, drums, metronome).
- Drone: A plug-in that loads into a DAW synchronized with Synfire. It is remote-controlled by Synfire to load guest plug-ins and play the MIDI output rendered by Synfire.
(this is not yet complete of course, but it should probably not get much longer)
I'll post occasional updates on progress here. Although MCP is not currently a priority, we already had a server skeleton at hand for experimenting and that was a very tempting distraction.
The mid term goal is to evaluate whether a natural language interface to the user manual can be done and if it's worth the effort. If the AI is able to explain step-by-step instructions for any imaginable task a user may want to accomplish, that'd be a great win.
I don't know how many users might be willing to add a $20/month Claude/ChatGPT/Gemini subscription to their music budget though. Unless they already have one anyway.
What do you think?
Sat, 2026-08-01 - 15:01 Permalink
“If the AI could explain step-by-step instructions for every conceivable task a user wants to perform, that would be a huge benefit.”
I use the ChatGPT-Go subscription for 8 euros and haven't hit a user limit yet, and Deepseek is free and can also be used for a long time.
I'm curious to see where this will lead!
The Go ChatGpt subscription doesn't include MCP support, but I think that's something that's more important to a developer.
Recently, I also considered building my own MCP server, but the program I wanted to use it for already offered this feature at the same cost
It’s a lot of hassle to try to build this myself, but given how professionally the software’s creator develops it, I won’t be able to match that level of quality.
So I can also use MCP in the software I use for my hobby if i want it.
Sat, 2026-08-01 - 23:13 Permalink
I would think most people would find an ai RAG model more use, trained on the manual and perhaps some of the sections in the forum like the tutorials and user tips. It would help people search the manual for something without having to be so precise with the search query. It might even be possible to deliver this model with synfire to be run locally as I can't see it being that large or require too much processing once trained. There are similar models that run on mobiles and low end machines like the raspberry pi.
An MCP server is an interesting idea, but depending on what it delivers most users once they've got beyond the basics, will find it more longwinded than using the menus, buttons and key short cuts. Experts probably use the key shortcuts most. Maybe if the MCP server can replace, drive or supplement the factories allowing the user to generate figures, harmonies and other parameters from text input that would be great. So typing something like "container B is the verse, container c is the chorus, can you make the verse sound more sad and the chorus more uplifting" and having it make changes would be useful.
If you do release an MCP server can you allow it to be used with local models?
Sun, 2026-08-02 - 11:32 Permalink
Yes, the chatbot can get a bit chatty unless it is briefed to stay brief (wow, two puns in one short sentence).
It's exciting to see it reason about the plugins, devices, and sounds I have installed, inferring that my current open project is "Fragile 1 (Circular)" with a "NI Noire" piano and "VSL Strings", that my AUX module in the global rack has a reverb loaded, all that. It even tells me that I'm currently looking at the "Structure View" and explains it to me (the MCP serves live context-sensitive information).
But all of that is obvious to the user anyway, or accessible through the help browser.
One benefit could be scripting: "Prepare this arrangement for chamber orchestra based on my VSL and SSO libraries" - it might add instruments, suggest and load plugins ... and then? Apart from the cool experience of instructing Synfire with natural language, there's little benefit. Dropping a few rack module presets is faster.
I understand that VSL with their MCP for VEP 8 have a much better use case. You might have multiple VEP servers in your LAN with hundreds of CCs to configure and routings to setup. Intelligent scripting can save a lot of time.
Synfre deliberately has only basic mixing capabilities with not that many options to configure. The only real chore is to build device descriptions from scratch. If AI could help with that, I'm all for it.
Also if a chatbot could figure out step-by-step instructions for any task on demand, that would be worth the effort. That would absolutely flatten the initial learning curve.
Yes, local models would work as well. The MCP runs on (http://127.0.0.1), open for any client with a token that Synfire generates.
Sun, 2026-08-02 - 12:16 Permalink
container B is the verse, container c is the chorus, can you make the verse sound more sad and the chorus more uplifting
Actually this is a good example of what Synfire should probably not (attempt to) do.
Emotion is carried by many dimensions (harmony, rhythm, sounds, tempo, melody, voice), there is no obvious parameter or element that could be tuned to move a piece of music on that spectrum. What this kind of prompt asks for is a complete rebuild of everything. Then we are in Suno territory, i.e. it's no longer about making music, but more like navigating a random playlist.
What you can do now already is ask your favorite LLM: "What makes a song feel sad versus uplifting? Name at least four distinct musical properties that influence emotion" - and then you simply tinker a bit with those parameters ;-)
Sun, 2026-08-02 - 14:49 Permalink
did just that :
Great question — emotion in music comes from several dimensions working together (often reinforcing each other, sometimes deliberately clashing). Here are six of the most influential ones, contrasted along the sad ↔ uplifting axis:
| Dimension | Tends toward SAD | Tends toward UPLIFTING | Why it works |
|---|---|---|---|
| Tempo | Slow (roughly 60–80 BPM), sometimes rubato/dragging | Faster (110–140+ BPM), steady drive | Tempo maps onto physiological arousal — a slow pulse mirrors a resting or heavy body; a quick one mirrors excitement and movement. |
| Mode / Harmony | Minor mode, unresolved or suspended chords, chromaticism, slow harmonic rhythm | Major mode, clear resolutions (V→I), consonance | Minor thirds and unresolved tension read as "unsettled/mournful"; major intervals and resolution read as "arrival/safety." |
| Melody (contour & intervals) | Descending contours, small/narrow steps, sighing figures, low register | Ascending contours, wider leaps (esp. rising 4ths/5ths/octaves), higher register | We hear rising pitch as rising energy/hope and falling pitch as deflation — it echoes the shape of the speaking voice in those emotions. |
| Rhythm | Sparse, syncopation used for hesitation, long note values, uneven/limping feel | Regular, propulsive, groove-locked, shorter active values | Predictable, driving rhythm invites the body to move and feel confident; irregular or sparse rhythm creates instability or stillness. |
| Timbre / Instrumentation | Soft, dark, breathy or muted tones (cello, solo piano, reverb-drenched pads) | Bright, full, resonant (horns, distorted guitars, layered vocals, bells) | Brightness of the sound spectrum correlates with perceived energy; darker, thinner timbres feel intimate or bleak. |
| Dynamics / Articulation | Quiet, legato, gentle swells, gradual decay | Loud, accented, staccato punches, big contrasts | Volume tracks intensity; smooth legato feels tender or weary, while crisp accents feel energetic and celebratory. |
A few practical notes for songwriting:
These dimensions are additive but not equal in weight. Mode and tempo are the two heaviest levers — a major-key song at 130 BPM will resist sounding sad even with dark lyrics. If you want emotional nuance rather than a blunt signal, keep most dimensions aligned but pull one against the grain. Some of the most affecting songs are bittersweet: a bright, uplifting arrangement over a minor melody, or upbeat production under a descending, resigned vocal line. That tension is where a lot of memorable songwriting lives.
Also worth remembering: context and expectation matter as much as the raw parameters. A relative rise after a long descent lands as uplift even in a minor key; a sudden silence can feel more devastating than any note. Emotion comes from the change over time, not just the static settings.
If you tell me the specific mood you're chasing (say, "hopeful but wistful" or "triumphant"), I can suggest a concrete combination of these levers to aim for.
Fri, 2026-08-07 - 21:35 Permalink
I don't know if MCP can handle screen/mouse/KB interactions but this ability would give complete control of synfire to an AI agent, so we could only talk music to the AI with a distant eye to synfire. The availability of MCP for sound library is also a great thing for transparent AI control.
edit: just to be dreamy : a linux version of Synfire running on the nvidia small AI machine dgxspark (1000TFLOP/128GB RAM 4000$ : can run midsize model like deepseek V4 v731) would allow full local AI music :)
Sun, 2026-08-09 - 11:46 Permalink
AI music requires domain specific transformers that are not readily available