Skip to content
←Back to Applications

APPLICATION

Speech & AudioMediaTop

Eleven Music v2.5: the closed-source row that turns music generation into a callable pipeline

eleven-music

The music generation model from ElevenLabs, currently at v2.5, released 2026-09-11 with the release post last updated 2026-09-20. It sits at the engineering-side top row for music in the audio domain, which contrasts with rather than duplicates the Suno entry already catalogued here: the value of Suno concentrates inside the product, a Studio multi-track timeline, Custom Models, up to twelve stems and MIDI export, while Eleven Music splits comparable capability into endpoints, namely compose, stream, a structured composition plan, scoring an uploaded video, uploading existing audio, stem separation, finetunes and section level inpainting, plus a marketplace where creators license tracks to each other. Choosing between them therefore needs no audio quality comparison, only one question: is this track something a person sits and adjusts, or something a pipeline requests in volume. The API surface is nine endpoints rather than one generate call, and two details matter for procurement: both compose and stem-separation take a sign_with_c2pa flag that applies to mp3 output so outgoing files can carry content credentials, and output format is tied to subscription tier, with mp3_44100_192 requiring Creator or above and pcm_44100 requiring Pro or above on stem-separation and video-to-music while compose goes up to mp3_48000_320, so the threshold for lossless differs per endpoint. The trap worth memorising is that v2.5 is already the interface default yet the model_id enum of music_v1, music_v2 and music_v2_5 still defaults to music_v1, and output_format=auto resolves per model to mp3_44100_128 on v1 and mp3_48000_192 on v2, so a minimal call that omits model_id silently gets the oldest generation at a lower bitrate; pin both in production code and name mp3_48000_320 when 320kbps is required. The two plan schemas are not interchangeable and the wrong pairing is a hard error: music_v1 takes MusicPrompt while music_v2 and music_v2_5 take CompositionPlan, whose chunks carry a text field with square bracket section names, lyric lines and curly brace inline directions. On the control surface prompt and composition_plan are mutually exclusive, and the request takes music_length_ms from 3000 to 600000, that is three seconds to ten minutes, plus force_instrumental, finetune_id and seed. Rights and commercial use are the most structured part of this line: a multi-year agreement with Universal Music Group was announced alongside v2.5 and the vendor states it is separate from Music 2.5; every track is yours on every plan including Free, Free allows commercial use provided ElevenMusic is credited, lossless downloads are capped at five per day on Free and 400 per month on Pro, tracks built on another artist song through Audio Reference cannot be downloaded, the Marketplace sells licences by usage type with creator earnings starting at 25 percent, and no licence permits distribution to streaming platforms such as Spotify. The official evidence for v2.5 over v2 is a self-run blind test in which v2.5 won the majority of 47,885 paired takes with the widest gap in vocal-led and acoustic-heavy genres. Boundaries: the API is paid-subscription only; vocals are documented for English, Spanish, German and Japanese with no Mandarin, so a Chinese language song is more practical on YuE2 or Suno; seed does not guarantee reproducibility; and it is closed source with no weights, so the self-hostable alternatives here are YuE2 for music and VoiceStudio or Kokoro-82M for speech. Graded C (vendor claim): quality was not recomputed and no blind listening test was run, but the API contract is verifiable documentation and every clause of it is listed in the body.

C
CONFIDENCE
Vendor Claim
Official model card or keynote only, no independent re-test
10 min
KEY METRIC
API per-track duration cap (music_length_ms 3000-600000)
Vendor Claim · 2026-09
MATURITY
Production
research → demo → product → production
Editor's take

Graded C (vendor-claim) because the only quantitative quality evidence citable is a vendor-run blind test: the same prompt rendered as two takes and compared as pairs, with v2.5 preferred in the majority of 47,885 pairs and the widest gap in vocal-led and acoustic-heavy genres. The sample size is respectable, but it is a first-party A/B with no third party benchmark reading to cross-check, and no independent blind listening test was run, so the contract puts it at C. What does hold up at grade B is a different thing: the API contract is verifiable documentation. music_length_ms from 3000 to 600000, three model_id values with music_v1 as the default, output_format=auto resolving to mp3_44100_128 on v1 models and mp3_48000_192 on v2 models, the video-to-music ceilings of 10 files and 200MB and 600 seconds, and the tier gates on stem-separation where 192kbps mp3 needs Creator while pcm_44100 needs Pro. None of that is evidence about sound. It is the boundary a reader can write code against, and listing every one of those in the body is precisely how a C row stays useful.

Its role at the top of the audio ladder is the engineering-side representative for music, which contrasts with rather than duplicates Suno, the other closed-source music row. Suno pulls the back half of music production into one product, a Studio multi-track timeline, Custom Models, up to twelve stems and MIDI export, and its value is a person finishing a song in an interface. Eleven Music splits comparable capability into endpoints, compose, detailed, stream, composition-plan, video-to-music, upload, stem-separation, finetunes and inpainting, and its value is a program finishing a song. Choosing between them therefore does not require comparing audio quality, only asking one question: is this track something a person sits and adjusts, or something a pipeline requests in volume. Automatic video scoring, podcast intros, batch game music and an agent scoring its own output are the second case; an independent musician reworking a chorus is the first.

The third reason has nothing to do with music and matters most to procurement: this is the line where rights are documented in the most structured way. A multi-year strategic agreement with Universal Music Group spanning licensing and product development, which the vendor states is separate from Music 2.5. Ownership that attaches to the track, with terms changes applying only to new tracks. Commercial use on Free provided ElevenMusic is credited. Downloads blocked on tracks built from another artist song. A marketplace that sells licences in four usage types, Social Media, Paid Marketing, Offline and Enterprise, with creator earnings starting at 25 percent. A prohibition, under every licence, on distributing to streaming platforms such as Spotify and Apple Music. And a sign_with_c2pa flag on both compose and stem-separation. When an enterprise adopts generative music the thing that actually goes wrong is the rights chain and content provenance, not a 3dB quality difference, and this line turns the rights chain into terms you can read. That is scarce among closed-source vendors.

Four practical boundaries. First, do not trust SDK defaults: v2.5 is the default in the product interface while the API default for model_id is still music_v1, and combined with output_format=auto that yields the oldest model at a lower bitrate with nothing in the response to flag it. Pin both the model id and the output format in production code. Second, the plan schemas are not interchangeable: MusicPrompt feeds v1 only and CompositionPlan feeds v2 and v2_5 only, so the wrong pairing is a hard error, and section durations can be relaxed on v1 with respect_sections_durations=false while v2 models always enforce them. Third, Mandarin vocals are not in the documented set, which lists English, Spanish, German and Japanese; for a Chinese-language song move to YuE2 or Suno rather than forcing it through a prompt. Fourth, the docs lag the release: the product page still names v2 as the interface default and still describes 44.1kHz exports at 128 to 192kbps, while the release post from the same week says v2.5 is default and that every plan including Free gets lossless downloads. Where they disagree, the parameter enums in the API reference are the authority.

Music GenerationClosed SourceAPI FirstContent Credentials
ElevenLabsOfficial site9 min read912

The problem it solves: turning song generation into a music pipeline a program can call

Eleven Music is the music generation model from ElevenLabs, currently at v2.5, released 2026-09-11 (release post last updated 2026-09-20). Its division of labour with Suno, the other closed-source top row for music in the audio domain, is not about sound quality but about programmability. The value of Suno concentrates inside the product: a Studio multi-track timeline, Custom Models, up to twelve stems and MIDI export. Eleven Music splits the same set of capabilities into endpoints: compose, stream, a structured composition plan, scoring an uploaded video, uploading existing audio, stem separation, finetunes, section level inpainting, plus a marketplace where creators license tracks to each other. That is why we list it as the engineering-side top row for music in the audio domain: if the job is to embed music generation into your own product or agent workflow, this is the most complete closed-source surface we have found.

The official claim for v2.5 over v2 is better audio quality and better prompt adherence with no change in capability set, so Audio Reference, composition plans and inpainting all carry over. The evidence offered is a self-run blind test: the same prompt rendered as two takes, one from each model, compared as pairs, with v2.5 preferred in the majority of 47,885 pairs. The gap is widest in vocal-led and acoustic-heavy genres, named as R&B, soul, hip hop, rock, metal, orchestral and cinematic. That is a vendor A/B rather than a third party leaderboard and we did not recompute it, which is what sets the confidence grade for this row.

The API surface: nine endpoints, not one generate call

EndpointMethod and pathWhat it is for
Compose musicPOST /v1/musicOne song from a prompt or a composition plan, returned as an audio file stream
Compose music with detailsPOST /v1/music/detailedSame generation with a structured response carrying the song id and metadata that inpainting needs
Stream musicPOST /v1/music/stream (plus a detailed variant)Audio while it generates, for interactive products and for Music as a Flows node
Create composition planPOST /v1/music/composition-planExpands one prompt into a JSON plan you can edit before spending a generation on it
Video to MusicPOST /v1/music/video-to-musicScore uploaded video: up to 10 files concatenated in order, 200MB combined, 600 seconds total, with an optional description under 1000 characters and up to 10 style tags
Upload Music / Stem SeparationPOST /v1/music/upload, POST /v1/music/stem-separationSplit existing audio into stems and return a ZIP; the docs warn explicitly about high latency on long files
Finetunes/v1/music/finetunes (list, create, get and friends)Train a private style variant on your own original audio and pass finetune_id at generation time

Two details are easy to miss and matter for procurement. First, C2PA signing: both compose and stem-separation take a sign_with_c2pa flag that applies to mp3 output, so outgoing files can carry content credentials. Where AI-generated music has to survive a platform review step, that is a hard requirement rather than a nice extra. Second, output format is tied to subscription tier: on stem-separation and video-to-music the docs state that mp3_44100_192 requires Creator or above and pcm_44100 requires Pro or above, while compose goes up to mp3_48000_320. Getting lossless therefore has a different threshold on each endpoint, so check per endpoint instead of assuming one product-wide rule.

The control surface: the composition plan is the most valuable part of this product line

Prompt-only generation exposes music_length_ms (3000 to 600000, that is 3 seconds to 10 minutes; omit it and the model picks a length from the prompt), force_instrumental (true guarantees an instrumental), finetune_id, store_for_inpainting and seed. prompt and composition_plan are mutually exclusive, and seed and force_instrumental only work with a prompt.

Delivery-grade arrangement goes through a plan, and there is one thing to read carefully: the two plan schemas are not interchangeable and using the wrong one is a hard error.

music_v1 takes MusicPromptmusic_v2 and music_v2_5 take CompositionPlan
Structural unitsections (SongSection)chunks
LyricsA separate lines field, at most 30 lines per section and 200 characters per lineWritten into text: a section name in square brackets such as [Verse 1], then lyric lines, with inline directions in curly braces such as {scratching}; same 30 line and 200 character limits
StyleGlobal positive_global_styles and negative_global_styles plus per-section positive and negative local stylesEach chunk carries positive_styles and negative_styles, and the docs note that the styles of the first chunk matter most because they set the tone and genre of the whole song
Section timingduration_ms 3000 to 120000, and respect_sections_durations=false relaxes it so the model can adjust individual sections for quality and latency while preserving the totalduration_ms 3000 to 120000, always enforced, and that flag is ignored
InpaintingSection level source_from with song_id, range and negative_rangesSame semantics through the inpainting guide

Two further controls live in the product rather than in the request body. Audio Reference takes an uploaded clip of roughly 30 seconds or less to guide a v2 or v2.5 generation, and every upload is screened for copyright compliance. The official wording is precise about what it does: it influences sound, production style, instrumentation, tempo and mood, and it does not copy or remix the uploaded audio. Finetunes come in two kinds, curated ones pretrained by ElevenLabs across global genres and custom ones trained on your own original audio. Together those two answer the question a prompt cannot, which is how to make your music keep sounding like yours.

The trap worth memorising: the API default model is still music_v1

In the product interface v2.5 is already the default for prompted and reference generation, but the model_id enum on POST /v1/music is music_v1, music_v2 and music_v2_5 with a default of music_v1. At the same time output_format=auto means pick the best format for the selected model, which resolves to mp3_44100_128 for v1 models and mp3_48000_192 for v2 models. Put together, a minimal call that omits model_id gets the oldest generation of the model at a lower bitrate, and nothing in the response says so. The official quickstart does pass model_id="music_v2_5" explicitly, yet plenty of real integrations get carried along by an SDK default. Make two things explicit in production code: the model id, and the output format, naming mp3_48000_320 when you need 320kbps instead of trusting auto.

A second caution is that the documentation lags the release. The product docs page still states that Music v2 is the default model in the Eleven Music interface while the 2026-09-11 release post states that v2.5 is now the default. On export, the product docs describe MP3 at 44.1kHz and 128 to 192kbps with other formats coming soon, while the same week post announces lossless downloads on every plan including Free. Where those conflict our rule is to trust the parameter enums in the API reference, because that is the surface the SDKs and the validation layer actually consume.

Rights and commercial use: a UMG agreement, tiered licences, and a marketplace starting at 25 percent

For enterprise procurement the primary risk in music generation is rights rather than audio quality, and this is the closed-source line where rights are documented in the most structured way.

  • A multi-year strategic agreement with Universal Music Group was announced alongside v2.5 and spans licensing and product development. The vendor states explicitly that this deal is separate from Music 2.5, so do not read a new model as a licensed catalogue.
  • Ownership and commercial use: every track you make is yours on every plan including Free. Free allows commercial use provided ElevenMusic is credited. Lossless downloads are capped at five per day on Free and 400 per month on Pro. The permissions attach to the track, so cancelling or downgrading does not retroactively change what you may do with tracks already created, and any future tightening of terms applies only to tracks made from that day onwards.
  • One hard block: tracks built on another artist song, meaning an Audio Reference to a third party work, cannot be downloaded. The vendor frames this as symmetric protection, the same rules that protect your music protect theirs.
  • Music Marketplace: creators publish songs they generated with ElevenLabs and buyers purchase a licence by usage type, which are Social Media (your own channels including monetized video, podcasts, personal sites), Paid Marketing (paid ads, client and agency work, commercial products), Offline (live events, trade shows, physical venues and installations) and Enterprise and Custom (TV, film, cinema, VOD and OTT streaming, large-scale distribution, radio, large studio games). Creator earnings start at 25 percent of the purchase price and pay out through the existing ElevenLabs payout system.
  • Two things are prohibited under every licence: distributing tracks to music streaming platforms such as Spotify, Apple Music or SoundCloud, and reselling, sublicensing or claiming ownership of the music.

Boundaries and failure modes

Six of them. First, the API is paid-subscription only, so the free tier can create in the web product but cannot drive a backend batch job. Second, the vocal language set is narrow: the documented languages are English, Spanish, German and Japanese, with no Mandarin vocals, so for a Chinese-language song the more practical rows in our audio domain are YuE2 (open weights, non-commercial licence) or Suno. Third, seed does not guarantee reproducibility: the official wording is that the same seed with the same parameters helps consistency but exact reproducibility is not guaranteed and outputs may change across system updates, and seed cannot be combined with prompt anyway. Projects that need a stable long-term sound should store the plan and the finished audio rather than replay parameters. Fourth, ten minutes is the ceiling for a single track, and going longer means segmenting and reassembling with inpainting or stems, which is a different workflow and a different latency budget. Fifth, the quality evidence is a vendor-run blind test with no verifiable third party benchmark reading, which is why confidence stays at C. Sixth, it cannot be self-hosted: closed source with no weights, so unreleased melodies and lyrics travel through their service, and data-sovereignty-sensitive work has to be assessed on that basis. The self-hostable alternatives in our audio domain are YuE2 for music and VoiceStudio or Kokoro-82M for speech.

More in Speech & Audio

4
Speech & AudioMediaTopC

Gemini 3.8 Flash TTS: turning director-level voice direction into API parameters

Google's speech generation model (gemini-3.8-flash-tts), positioned for studio-grade voice fidelity, expressive acting and long-form stability, the most overlooked being the last: audiobook failure is not an ugly sentence but timbre drift by hour three. The design separates scopes: text is strictly a script while performance direction travels on another channel and is never read aloud, fixing "(sighs)" that used to be spoken; round style goes into speech_metadata.style, word-level emotion into pipe tags. Four voice sources: prebuilt, an extended library, voice design IDs (describe it, then reuse) and replication IDs; design and replication arrived only in 3.8. Specs: 130 languages with automatic language detection, WAV by default at 24 kHz, at most 2 speakers per request and only with prebuilt voices. Billing is per audio token at 25 tokens per second, $0.50 input and $9.00 output per million through 2026-12-31 then $1.00/$18.00. Limits: 24 kHz is not mastering grade; replication compliance rests with the user. No speech board among the 12 synced here, so listed alongside ElevenLabs v3 without ranking. Graded C (vendor-stated): no intelligibility or independent blind listening test.

All supportedVoice design + cloning + multi-speakerVendor Claim · 2026-09
ProductGoogleSite
Gemini 3.8 Flash TTS: turning director-level voice direction into API parameters
Speech & AudioMediaTopA

VoiceStudio: the commercial voice stack moved onto local hardware and opened to agents by default

An open source, fully local voice workstation (debpalash/VoiceStudio, AGPL-3.0, Python plus Electron) positioned as the local alternative to ElevenLabs: voice cloning, voice design, video dubbing, dictation, transcription and audiobook creation, with a claimed 646 languages. Created on 2026-04-09 and at 42,831 stars and 5,005 forks on 2026-09-29. It sits at the top self-hosted row in the audio domain, not because any model behind it is stronger, but because the abstraction layer is useful. Open voice capability is scattered across a dozen repositories with their own weight formats, device requirements, cloning interfaces and licences, so assembling a pipeline of clone a voice, transcribe, align, dub and produce an audiobook spends most of its effort on glue. VoiceStudio collects 17 TTS and 11 ASR engines into one registry and reports per engine which devices it runs on (CUDA, MPS, CPU), whether it can clone, and its max_ref_seconds and ref_strategy, on top of a workflow that actually runs. The agent surface matters most for the catalogue: a local API and an MCP server ship as first class product features, the README includes an installation prompt ready to paste into Claude Code, Codex or Cursor, and npx skills add installs it as a skill, which means coding agents can call dubbing and transcription as tools rather than needing a human at the interface. The reference audio policy is equally rigorous, with a 75 second ceiling while how much of a longer clip reaches the model depends on the engine and is reported per engine, an automatic or saved transcript ignored beyond the limit, and a hard [clone_ref_too_long] error when sending more than 20 seconds plus transcript text to the default OmniVoice. One measurement point needs stating plainly: the official docs/benchmarks.md contains a real harness with per stage profiling, refusal to start when memory is insufficient, and a rule that only RTF warm and peak memory on named hardware and versions are accepted with no estimated numbers allowed, but the results column reads No verified rows yet. therefore attach no performance figure at all and the card carries only the star count verified ourselves through the GitHub API, which is what earns the A grade. That A asserts the number is real and nothing about being better than ElevenLabs. Four boundaries: AGPL-3.0 is strong copyleft with a network clause, so embedding it in a service offered to outsiders obliges you to release your own server side source, and closed commercial use means either process isolation calling only the local API under your own legal review or assembling the pipeline from Apache-2.0 pieces such as CosyVoice 3 or Kokoro; per-model licences are not settled by the wrapper, since the Breeze-TTS-2 weights behind audio.cpp are research and non-commercial, Supertonic-3 and PocketTTS need extra permissions and GPT-SoVITS requires its own server, so check every model before commercial use; the 646 language figure is the repository own claim and real coverage depends on the engine chosen, from 25 European languages on Parakeet through 50 plus on FunASR to the wider Whisper family, unverified language by language; and desktop is Electron only now, with 0.5.3 the last Tauri release, so existing users must migrate. It sits alongside Eleven v4 on the ladder rather than replacing it, one representing the quality ceiling and convenience of a closed commercial stack and the other the data sovereignty and engine substitutability of local self-hosting.

42,831GitHub stars (2026-09-29, verified here via GitHub API)Confirmed · 2026-09
Productdebpalash (open source)SiteRepo
VoiceStudio: the commercial voice stack moved onto local hardware and opened to agents by default
Video GenerationMediaTopC

Kling 3.0 series: native audio, 4K frames, and parameterised camera language as the moat

Kuaishou Kling was among the first Chinese video lines to reach a level where it can take commercial work, and its lasting strength is not raw text-to-video but image-to-video plus camera control: lock composition and character in the image stage, then let the video model interpolate from that fixed keyframe, with first/last-frame constraints and push/pull/pan/truck/orbit/follow parameters producing predictable clips. The 3.0 series (official release-note banner: API fully available) brings three substantive jumps - audio generated natively (5s/10s/12s audio tiers), native 2K/4K for both image and video, and storyboarding plus element reference turning multi-shot coherence from an editing problem into something the model controls; the line-up is 3.0 / 3.0 Omni / 3.0 Turbo, Kling Image 3.0 and -omni, Native 4K Video and Motion Control. The arena reading, stated plainly: image-to-video kling-v3-pro is #19 (Elo 1354), behind MiniMax h3, gemini-omni, wan3.0 and seedance-2.5. No authoritative release date for the 3.0 series could be obtained, so the metric date is recorded only at the verifiable month, 2026-09. Closed, vendor API only; not independently benchmarked, graded C.

3.0 / Omni / TurboCurrent flagship seriesVendor Claim · 2026-09
ProductKuaishouSite
Kling 3.0 series: native audio, 4K frames, and parameterised camera language as the moat
Speech & AudioMediaTopC

Eleven v3: the commercial audio stack that covers speaking, listening and singing

ElevenLabs' flagship speech synthesis model, officially its most emotive and expressive: 70+ languages, a 5,000-character request cap, dramatic delivery and natural multi-speaker dialogue, with a separate Text to Dialogue API making multi-character dialogue an endpoint, not a prompt trick. It sits at a top row in audio for the stack behind it: the only commercial one covering speaking, listening and singing, with four TTS tiers, three ASR tiers including a Medical build and two music models. The TTS tiers are mutually exclusive: v3 is most expressive but non-realtime at 5,000 characters, Flash v2.5 is lowest latency (about 75ms, 50% cheaper per character) and Multilingual v2 is most stable for long text at 29 languages. On ASR, Scribe v2 covers 90+ languages with diarisation for 32 speakers. Latency figures are model-side only, excluding network and upstream LLM time, not SLAs. Limits: you split long text and own continuity across the seams; closed and not self-hostable, a hard barrier where data sovereignty matters. No speech board among the boards synced here, so listed alongside Gemini 3.8 Flash TTS without ranking. Graded C (vendor-stated): no WER recomputation or independent blind listening test.

70+Language coverageVendor Claim · 2026-09
ProductElevenLabsSite
Eleven v3: the commercial audio stack that covers speaking, listening and singing

As an Amazon Associate, we earn from qualifying purchases.