Skip to content
←Back to Applications

APPLICATION

Speech & AudioMediaTop

Eleven v4: the generation that pushes cloning fidelity far enough to reproduce bad training data

elevenlabs-v4

The new generation speech synthesis model from ElevenLabs, shipping in two variants: Eleven v4 for highest quality and Eleven v4 Turbo for real time use at a median inference latency of about 100 ms, with an official footnote stating that application and network latency are excluded. The vendor positions it above v3 on output quality, voice accuracy, consistency, emotion, delivery, audio tags and language coverage, and recommends migrating. It sits at the top closed source voice row in the audio domain because this generation changed the optimisation target from sounding better to sounding more like the source: timbre, cadence, delivery and other mannerisms are reproduced together, and the vendor concedes that accuracy and personal preference are not always the same thing and that you may still prefer how a voice sounded in v3. The engineering consequence is hard, since the burden of data cleaning moves back to the user. The showcase samples are explicitly labelled raw, with no EQ, compression, normalisation, de-essing or plosive removal, and the vendor says that if you hear clipping or plosives they are very likely in the training data and the model simply captured them accurately. What is deliverable comes down to three numbers: about 100 ms on Turbo; 10,000 characters per request, roughly ten minutes of audio, against 5,000 on v3 so long form text carries twice as many segment seams; and a coverage claim of 90 plus languages against a FAQ that enumerates 87, a gap kept visible rather than smoothed away. Four boundaries matter. Cross language accent handling is a default behaviour change and not a toggle, so generating in a language that differs from the reference produces fluent native sounding speech in the target language, and the vendor calls a switchable version a research project with no timeline; any persona that depends on a native accent carried into a second language must be tested first. The control surface narrowed to Stability and Similarity, the Style and Speed sliders are gone, SSML is unsupported, and fine grained delivery moves to audio tags which the vendor admits are not perfect yet. Continuous model evolution is stated in writing, with training continuing after launch and behaviour possibly shifting over time, so teams treating a voice as a brand asset need periodic re-testing. Voice Design voices may also be less performative on v4. It is closed and not self-hostable, data must pass through ElevenLabs, and cloning compliance sits with the user; the self-hostable counterparts are CosyVoice 3, VibeVoice and Kokoro for speech and YuE2 for music under a non-commercial weight licence. This entry and the Eleven v3 already catalogued here are two generations of the same commercial stack rather than a replacement. Graded C (vendor claim): no verifiable third party benchmark, and no independent blind listening test.

C
CONFIDENCE
Vendor Claim
Official model card or keynote only, no independent re-test
~100ms
KEY METRIC
v4 Turbo median inference latency (model side, excl. app and network)
Vendor Claim · 2026-09
MATURITY
Production
research → demo → product → production
Editor's take

Graded C (vendor claim). ElevenLabs publishes no verifiable third party benchmark: there is no MOS figure, no speaker similarity number and no WER citable, only the vendor own generational comparison plus a set of showcase samples, and no independent blind listening test was run. Contract section 13.5 therefore forbids an upgrade to A. What makes this entry credible within that grade is something else: the vendor labels its showcase samples as raw, with no EQ, compression, normalisation, de-essing or plosive removal, and states outright that if you hear clipping, plosives, uneven EQ or volume jumps then those artefacts are very likely in the training data and v4 simply captured them accurately. Publishing the flaws of your own output for anyone to hear is unusual in an audio model release, and it is why this qualifies as the top closed source voice row rather than merely relaying the marketing claim.

The case for the top of the ladder is a generational step on the cloning fidelity axis, paired with a real time latency that is rare at this quality tier. The change from v3 to v4 is not that it sounds better but that it sounds more like the source: timbre, cadence, delivery and other mannerisms are reproduced together, and the vendor itself concedes that accuracy and personal preference are not always the same thing and that you may still prefer how a voice sounded in v3. That concession amounts to admitting the optimisation target of this generation is fidelity rather than pleasing the ear. The engineering consequence is hard: the burden of data cleaning moves back to the user, because reference recording quality now determines output quality directly instead of being masked by model side beautification, and the vendor changed its training data guidance accordingly (stick to a single speaking style, with an explicit note that the guidance may change). What is actually deliverable comes down to three numbers: median inference latency around 100 ms on the Turbo variant, with an official footnote stating that application and network latency are excluded; 10,000 characters per request, roughly ten minutes of audio; and a language claim of 90 plus against a FAQ that enumerates 87. For contrast, v3 allows 5,000 characters per request so long form text carries twice as many segment seams, and Flash v2.5 is faster at about 75 ms but covers only 32 languages with text normalisation off by default. The gap is kept between 90 plus and 87 visible rather than smoothing it, because a headline coverage claim and an enumerable list have always been two different numbers.

Four boundaries must be understood before migrating. Cross language accent handling is a default behaviour change, not a toggle: when the generated language differs from the reference voice language, v4 produces fluent native sounding speech in the target language instead of carrying the reference accent across, and the vendor says making it switchable is still a research project with no timeline. If a product persona depends precisely on a speaker carrying a native accent into a second language, as in character voiceover or interview reconstruction, this upgrade silently changes the output and must be tested or avoided by staying on v3. The control surface narrowed: only Stability and Similarity remain, the Style and Speed sliders are gone, SSML is unsupported, and fine grained delivery moves to inline audio tags which the vendor admits are not perfect yet and remain an active area of investment. For a pipeline built on SSML or on the Style slider this is migration cost, not upgrade benefit. Continuous model evolution is stated in writing: training continues after launch, behaviour may shift over time and periodic re-testing is recommended, so any product that treats a voice as a brand asset should put that on an operations calendar because reproducibility of output is not under your control. In addition Voice Design voices may be less performative on v4 in the vendor own words, so projects built on Voice Design should compare both generations before moving.

Then the one that cannot be routed around: closed and not self-hostable, data must pass through ElevenLabs, and the compliance burden for voice cloning sits with the user because capability is not authorisation. For teams whose data cannot leave their own infrastructure, or who need to hold the voice weights themselves, the self-hostable counterparts in the audio domain are CosyVoice 3, VibeVoice and Kokoro for speech, and YuE2 for music with the caveat that its weights are CC-BY-NC-4.0 and therefore non-commercial. One selection note: v4 and the Eleven v3 already catalogued in this catalogue are two generations of the same commercial stack, not a replacement. The vendor recommends migrating while also conceding that a few edge cases fit v3 better, so two coexisting rows on the ladder is the correct reading.

Text-to-SpeechVoice CloningRealtime TTSAudio Tags
ElevenLabsOfficial site5 min read796

The problem it solves: pushing clone fidelity far enough that it reproduces the flaws in your training data

Eleven v4 (eleven_v4) and Eleven v4 Turbo (eleven_v4_turbo) are the new generation of ElevenLabs speech synthesis models. The vendor states that v4 improves output quality, voice accuracy, consistency, emotion, delivery, audio tags and language coverage over v3, and recommends migrating. It sits on the same commercial audio stack as the Eleven v3 entry already in this catalogue rather than replacing it: v3 allows 5,000 characters per request while v4 allows 10,000, about ten minutes of audio, which halves the number of segmentation seams in long form work.

The most informative change in this generation is the direction of cloning fidelity. The documentation is unusually blunt: v4 reproduces the EQ, loudness, plosives, sibilance and volume fluctuation present in the reference recording, and the showcase samples are raw with no post-processing at all, meaning no EQ, compression, normalisation, de-essing or plosive removal. The vendor warns that recurring issues heard in those samples are most likely present in the training data for that voice and the model simply captured them accurately. The engineering consequence is hard: v4 pushes data cleaning back onto the integrator, because reference recording quality now determines output quality directly instead of being masked by model smoothing. The accompanying guidance changed as well, and currently recommends a single speaking style in training audio, with the vendor stating it is still experimenting and that this guidance may change.

Cross-language accent handling: a deliberate behaviour change that must be tested before migrating

This is the largest behavioural difference from every older model. When the generated language matches the language of the reference voice, the original accent is preserved exactly as before. When the two differ, v4 produces fluent natural speech in the target language rather than carrying the reference accent across. Cloning a Korean speaker and generating English yields natural English rather than English spoken with a Korean accent. That makes a single voice usable across every supported language and is the most practical gain in this generation for dubbing and multilingual voice agents.

It is, however, a default rather than an option. The vendor says it is exploring turning it into a toggle, describes that as a research project and gives no timeline. Products whose persona depends on a voice carrying its native accent into another language, such as character dubbing or interview reconstruction, therefore have to test v4 directly before migrating and may need to stay on v3. This site calls the change out separately because it belongs to the class of upgrades that silently alter output rather than purely improving it.

Two variants and a narrower control surface

ModelPositioningLatency (model side)Chars per requestLanguages
Eleven v4Highest quality: content creation, audiobooks, character voiceoverNon-realtime tier10,000, about ten minutesVendor states 90+, the FAQ enumerates 87
Eleven v4 TurboRealtime: conversational agents, interactive voiceMedian ~100ms-Same family coverage
Eleven Flash v2.5 (previous realtime tier)Lowest latency and lowest price~75ms40,00032
Eleven v3 ConversationalPrevious expressive realtime tier~280ms-70+

The control surface narrowed in this generation. Only two settings remain, Stability, which controls how consistent delivery stays across generations, and Similarity, which controls adherence to the reference voice at some cost to naturalness. The Style and Speed sliders are not available and SSML is not supported. In their place are audio tags such as [whispering], [shouting] and [laughing], which the vendor says v4 handles with nuance beyond previous models while also stating that they are not perfect yet and remain an active area of investment. This is a paradigm shift from continuous sliders to inline markers, and for a pipeline built on SSML or the Style slider it is a migration cost rather than an upgrade benefit.

Both the ~100ms and ~75ms figures carry the official footnote: they exclude application and network latency. In a real voice agent the upstream LLM usually dominates end to end latency by a wide margin, so these numbers are not SLAs. Flash v2.5 also disables text normalisation by default to protect latency, with an Enterprise-only apply_text_normalization flag, so the correct pattern on a low latency tier is to have the upstream LLM normalise numbers and symbols first.

Limits: the model keeps moving, Voice Design got weaker, closed and not self-hostable

  • The vendor states explicitly that the model will keep evolving. Training continues after launch and behaviour may shift over time, with a recommendation to re-test your use case periodically. For products where a voice is a brand asset this belongs in the operations calendar, because output reproducibility is not under your control.
  • Voices created with Voice Design may be less performative on v4 than on earlier models, in the words of the documentation. Optimising for faithful cloning of a real voice is not the same as optimising for designing a voice from a description, so Voice Design projects should compare both generations before migrating.
  • Two cloning paths. Instant Voice Cloning takes a one to two minute sample and is usable within seconds with no separate training step; Professional Voice Cloning trains a dedicated model on longer recordings and is fully supported on v4, with existing PVCs fine-tunable for v4 from Voice Lab.
  • Closed and not self-hostable, so all audio passes through ElevenLabs and the compliance burden for voice cloning stays with the integrator, since capability is not authorisation. The self-hostable counterparts in this domain are CosyVoice 3, VibeVoice and Kokoro for speech and YuE2 for music under a non-commercial licence.

More in Speech & Audio

4
Speech & AudioMediaTopC

Gemini 3.8 Flash TTS: turning director-level voice direction into API parameters

Google's speech generation model (gemini-3.8-flash-tts), positioned for studio-grade voice fidelity, expressive acting and long-form stability, the most overlooked being the last: audiobook failure is not an ugly sentence but timbre drift by hour three. The design separates scopes: text is strictly a script while performance direction travels on another channel and is never read aloud, fixing "(sighs)" that used to be spoken; round style goes into speech_metadata.style, word-level emotion into pipe tags. Four voice sources: prebuilt, an extended library, voice design IDs (describe it, then reuse) and replication IDs; design and replication arrived only in 3.8. Specs: 130 languages with automatic language detection, WAV by default at 24 kHz, at most 2 speakers per request and only with prebuilt voices. Billing is per audio token at 25 tokens per second, $0.50 input and $9.00 output per million through 2026-12-31 then $1.00/$18.00. Limits: 24 kHz is not mastering grade; replication compliance rests with the user. No speech board among the 12 synced here, so listed alongside ElevenLabs v3 without ranking. Graded C (vendor-stated): no intelligibility or independent blind listening test.

All supportedVoice design + cloning + multi-speakerVendor Claim · 2026-09
ProductGoogleSite
Gemini 3.8 Flash TTS: turning director-level voice direction into API parameters
Speech & AudioMediaTopA

VoiceStudio: the commercial voice stack moved onto local hardware and opened to agents by default

An open source, fully local voice workstation (debpalash/VoiceStudio, AGPL-3.0, Python plus Electron) positioned as the local alternative to ElevenLabs: voice cloning, voice design, video dubbing, dictation, transcription and audiobook creation, with a claimed 646 languages. Created on 2026-04-09 and at 42,831 stars and 5,005 forks on 2026-09-29. It sits at the top self-hosted row in the audio domain, not because any model behind it is stronger, but because the abstraction layer is useful. Open voice capability is scattered across a dozen repositories with their own weight formats, device requirements, cloning interfaces and licences, so assembling a pipeline of clone a voice, transcribe, align, dub and produce an audiobook spends most of its effort on glue. VoiceStudio collects 17 TTS and 11 ASR engines into one registry and reports per engine which devices it runs on (CUDA, MPS, CPU), whether it can clone, and its max_ref_seconds and ref_strategy, on top of a workflow that actually runs. The agent surface matters most for the catalogue: a local API and an MCP server ship as first class product features, the README includes an installation prompt ready to paste into Claude Code, Codex or Cursor, and npx skills add installs it as a skill, which means coding agents can call dubbing and transcription as tools rather than needing a human at the interface. The reference audio policy is equally rigorous, with a 75 second ceiling while how much of a longer clip reaches the model depends on the engine and is reported per engine, an automatic or saved transcript ignored beyond the limit, and a hard [clone_ref_too_long] error when sending more than 20 seconds plus transcript text to the default OmniVoice. One measurement point needs stating plainly: the official docs/benchmarks.md contains a real harness with per stage profiling, refusal to start when memory is insufficient, and a rule that only RTF warm and peak memory on named hardware and versions are accepted with no estimated numbers allowed, but the results column reads No verified rows yet. therefore attach no performance figure at all and the card carries only the star count verified ourselves through the GitHub API, which is what earns the A grade. That A asserts the number is real and nothing about being better than ElevenLabs. Four boundaries: AGPL-3.0 is strong copyleft with a network clause, so embedding it in a service offered to outsiders obliges you to release your own server side source, and closed commercial use means either process isolation calling only the local API under your own legal review or assembling the pipeline from Apache-2.0 pieces such as CosyVoice 3 or Kokoro; per-model licences are not settled by the wrapper, since the Breeze-TTS-2 weights behind audio.cpp are research and non-commercial, Supertonic-3 and PocketTTS need extra permissions and GPT-SoVITS requires its own server, so check every model before commercial use; the 646 language figure is the repository own claim and real coverage depends on the engine chosen, from 25 European languages on Parakeet through 50 plus on FunASR to the wider Whisper family, unverified language by language; and desktop is Electron only now, with 0.5.3 the last Tauri release, so existing users must migrate. It sits alongside Eleven v4 on the ladder rather than replacing it, one representing the quality ceiling and convenience of a closed commercial stack and the other the data sovereignty and engine substitutability of local self-hosting.

42,831GitHub stars (2026-09-29, verified here via GitHub API)Confirmed · 2026-09
Productdebpalash (open source)SiteRepo
VoiceStudio: the commercial voice stack moved onto local hardware and opened to agents by default
Video GenerationMediaTopC

Kling 3.0 series: native audio, 4K frames, and parameterised camera language as the moat

Kuaishou Kling was among the first Chinese video lines to reach a level where it can take commercial work, and its lasting strength is not raw text-to-video but image-to-video plus camera control: lock composition and character in the image stage, then let the video model interpolate from that fixed keyframe, with first/last-frame constraints and push/pull/pan/truck/orbit/follow parameters producing predictable clips. The 3.0 series (official release-note banner: API fully available) brings three substantive jumps - audio generated natively (5s/10s/12s audio tiers), native 2K/4K for both image and video, and storyboarding plus element reference turning multi-shot coherence from an editing problem into something the model controls; the line-up is 3.0 / 3.0 Omni / 3.0 Turbo, Kling Image 3.0 and -omni, Native 4K Video and Motion Control. The arena reading, stated plainly: image-to-video kling-v3-pro is #19 (Elo 1354), behind MiniMax h3, gemini-omni, wan3.0 and seedance-2.5. No authoritative release date for the 3.0 series could be obtained, so the metric date is recorded only at the verifiable month, 2026-09. Closed, vendor API only; not independently benchmarked, graded C.

3.0 / Omni / TurboCurrent flagship seriesVendor Claim · 2026-09
ProductKuaishouSite
Kling 3.0 series: native audio, 4K frames, and parameterised camera language as the moat
Speech & AudioMediaTopC

Eleven v3: the commercial audio stack that covers speaking, listening and singing

ElevenLabs' flagship speech synthesis model, officially its most emotive and expressive: 70+ languages, a 5,000-character request cap, dramatic delivery and natural multi-speaker dialogue, with a separate Text to Dialogue API making multi-character dialogue an endpoint, not a prompt trick. It sits at a top row in audio for the stack behind it: the only commercial one covering speaking, listening and singing, with four TTS tiers, three ASR tiers including a Medical build and two music models. The TTS tiers are mutually exclusive: v3 is most expressive but non-realtime at 5,000 characters, Flash v2.5 is lowest latency (about 75ms, 50% cheaper per character) and Multilingual v2 is most stable for long text at 29 languages. On ASR, Scribe v2 covers 90+ languages with diarisation for 32 speakers. Latency figures are model-side only, excluding network and upstream LLM time, not SLAs. Limits: you split long text and own continuity across the seams; closed and not self-hostable, a hard barrier where data sovereignty matters. No speech board among the boards synced here, so listed alongside Gemini 3.8 Flash TTS without ranking. Graded C (vendor-stated): no WER recomputation or independent blind listening test.

70+Language coverageVendor Claim · 2026-09
ProductElevenLabsSite
Eleven v3: the commercial audio stack that covers speaking, listening and singing

As an Amazon Associate, we earn from qualifying purchases.