# Speech Enhancement

> Speech enhancement removes noise and echo or rebuilds a voice in studio quality. Methods, tools, and the risk that a generative enhancer changes words.

Source: https://metavert.io/speech-enhancement  
Published: 2026-10-20  
Updated: 2026-10-20

**Speech enhancement** is the use of signal processing and machine learning to make recorded speech clearer: removing background noise, echo and room reverberation, separating a voice from music or other voices, and in its newest form regenerating the voice as if it had been recorded in a studio. It's a staple of creator tools, because most video and podcast audio is recorded in kitchens, cars and conference rooms rather than booths. It's also one of the places where AI quietly changes the content of a recording, which makes it worth understanding what each kind of enhancement actually does.

### Three Kinds of Enhancement

**Noise and echo suppression** keeps the original recording and removes what isn't speech. Classic methods estimated a noise profile and subtracted it; modern ones use neural networks that predict, for each moment and frequency band, how much of the signal is voice, and attenuate the rest. These systems run in real time on ordinary hardware and are built into video-call software. Because they only remove energy, they can make a voice sound thin or watery when pushed hard, but they don't add sounds that weren't there.

**Voice isolation and source separation** splits a recording into stems, such as voice, music and effects, so the voice can be kept and the rest lowered or discarded. It's the tool for a speaker recorded over music or at a loud event.

**Generative restoration** goes further. Instead of cleaning the recorded waveform, a generative model listens to the degraded speech and produces a new, clean rendition: full bandwidth, no room sound, studio presence. The results can be striking on poor recordings, because the model can restore frequencies a phone microphone never captured. But the output is a new performance synthesized from the old one.

### The Tools

[Open-source models](https://metavert.io/open-weight-models) cover the first and third kinds, though both of the best-known projects have been quiet for some time. DeepFilterNet, a real-time noise-suppression model published at ICASSP 2022 with later versions in 2022 and 2023, is open source; its latest release, 0.5.6, dates from August 2023. Resemble Enhance, released by Resemble AI in December 2023, has two stages: a denoiser that cleans the recording, and a generative enhancer, based on conditional flow matching, that restores distortion and extends bandwidth. Both still work, but neither has had a stable release since 2023.

Commercial tools span all three kinds:

- **[Adobe](https://metavert.io/adobe) Podcast Enhance Speech** is the best-known "make it sound like a studio" tool. Version 2 (November 2024) added a strength control, and a March 2026 update added source separation, with downloadable speech, noise and music stems.
- **[ElevenLabs](https://metavert.io/elevenlabs) Voice Isolator** (July 2024, with an API) separates speech from background noise, music and ambient sound.
- **DaVinci Resolve Studio** has included an AI Voice Isolation track effect since version 18.1 (November 2022).
- **[NVIDIA](https://metavert.io/nvidia) Broadcast** added Studio Voice, which makes a webcam microphone sound like a studio microphone, as a beta in version 2.0 (January 2025), running locally on RTX graphics cards.
- **Descript Studio Sound** applies AI voice enhancement inside Descript's editor.

### The Risk of Changed Words

A generative enhancer is, in effect, a speech synthesizer conditioned on a noisy recording. Where the input is clear, it reproduces what was said. Where the input is ambiguous, buried in noise, clipped or mumbled, it has to fill the gap, and it fills it with something plausible. That can mean a slightly different vowel, a changed consonant, a dropped word or an invented syllable, delivered in the speaker's own voice with full confidence. Researchers now study this directly. A 2025 paper accepted at AAAI 2026 describes two kinds of hallucination in generative speech enhancement: *linguistic* errors, where the spoken content changes, and *acoustic* errors, where the speaker's voice characteristics change, both worse at low signal-to-noise ratios. Users have reported the same thing in practice, including enhanced dialogue that contained words nobody said.

For entertainment content this may be acceptable. For interviews, journalism, legal or medical recordings, and anything where a person's exact words matter, it is a real problem, and one the speaker may never notice. The same concern applies to [speech recognition](https://metavert.io/speech-recognition), where models are known to produce fluent text that was never said, a form of [AI hallucination](https://metavert.io/ai-hallucinations).

### Using Enhancement Safely

- **Keep the original.** Treat enhanced audio as a new asset alongside the source, never a replacement, so the edit can switch back. This is [non-destructive editing](https://metavert.io/non-destructive-editing) applied to audio.
- **Use the least invasive tool that works.** Suppression first; generative restoration only when the recording is otherwise unusable.
- **Verify the words.** Transcribe the original and the enhanced version and compare them. Wherever the words differ, listen, and prefer the less processed take for that stretch.
- **Disclose when it matters.** If a voice has been substantially regenerated, that is synthetic media, and some contexts call for saying so; see [AI content labeling](https://metavert.io/ai-content-labeling).
- **Mind the level.** Enhancement changes loudness; re-measure before delivery (see [loudness normalization](https://metavert.io/loudness-normalization)).

### Where It Fits

Speech enhancement sits between capture and editing in modern [AI video editing](https://metavert.io/ai-video-editing) pipelines, usually as an optional step per speaker. It increasingly overlaps with [text-to-speech](https://metavert.io/text-to-speech) and voice cloning, since the most powerful restorers and the best synthetic voices use the same generative models. That overlap is the reason the line between "cleaned up" and "re-voiced" deserves an explicit check rather than an assumption.

## Related Topics

- [Speech and Voice AI](https://metavert.io/speech-voice-ai) — the wider field
- [Generative Audio](https://metavert.io/generative-audio) — the models behind generative restoration
- [Speech Recognition](https://metavert.io/speech-recognition) — transcription used to verify enhanced audio
- [Text-to-Speech](https://metavert.io/text-to-speech) — synthetic voices built on the same technology
- [Audio Ducking](https://metavert.io/audio-ducking) — mixing the cleaned voice with music
- [Loudness Normalization](https://metavert.io/loudness-normalization) — leveling after cleanup
- [Deepfakes](https://metavert.io/deepfakes) — when re-voicing becomes impersonation

## Further Reading

- [arXiv:2511.13300 (PASE)](https://arxiv.org/abs/2511.13300) — a paper accepted at AAAI 2026 on linguistic and acoustic hallucinations in generative speech enhancement
- [DeepFilterNet](https://github.com/Rikorose/DeepFilterNet) — open-source real-time noise suppression
- [Resemble Enhance](https://github.com/resemble-ai/resemble-enhance) — open-source denoiser plus generative enhancer
- [What's new in Adobe Podcast, March 2026](https://podcast.adobe.com/en/guides/whats-new-in-adobe-podcast-march-2026) — Adobe, including source separation in Enhance Speech
- [Voice Isolator](https://elevenlabs.io/docs/capabilities/voice-isolator) — ElevenLabs documentation