Reply to thread

Message: [QUOTE="Bombastus, post: 92131, member: 2178"] Singing voice conversion systems are built to change a singer's timbre while preserving the source melody and lyrics. The result is not a fresh performance assembled from nothing. Much of what you hear can still come from the original singer's delivery. A typical conversion starts with sung audio and tries to separate singer identity from the musical information worth keeping. Lyrics, pitch movement, rhythm, and parts of the source phrasing can survive while the system reconstructs the vocal with a different target timbre. The distinction matters when separating [B][URL='https://goldmidi.com/community/threads/personalized-vocals-do-not-mean-artist-voice-cloning.77087/']personalized vocals from artist voice cloning[/URL][/B] because changing vocal identity does not automatically replace the underlying performance. [HEADING=2]The source performance carries most of the musical intent[/HEADING] Melody is one of the clearest things a singing voice conversion system tries to preserve. Many systems extract the source fundamental frequency, or F0, then feed that contour back into the converted vocal alongside linguistic content and a target-singer representation. If the source slides late into a note or bends upward, some of that pitch behavior can remain in the output. Rhythm can survive in much the same way. The timing of syllables, held vowels, consonant placement, pauses, and note lengths often comes from the source recording rather than the target singer. A converted vocal may sound like the target person in timbre while still carrying the source singer's timing choices. Expression is harder to separate neatly. Older and newer systems can preserve aspects of source prosody because pitch, timing, loudness, and vocal content are entangled with the performance you supplied. Changing the timbre does not guarantee a convincing imitation of how the target singer would personally phrase the same line. Audio-to-audio singing conversion also differs from generating a new sung performance from a score. Conversion begins with an existing interpretation and keeps substantial musical structure from it. The target voice changes the identity layer, while the source recording still supplies much of the timing and melodic movement. [HEADING=2]Target timbre is replaced without cleanly erasing the source[/HEADING] The core job is usually to make the output sound as though it came from the target singer while keeping the song intact. Modern systems often encode linguistic content, melody, and singer identity through separate representations, then combine them during reconstruction. In practice, those representations are not perfectly independent. Source identity can leak into features that were supposed to contain only content. A converted vocal may carry traces of the original singer's timbre even when the target embedding is strong, producing a voice that sits awkwardly between both people. [B][URL='https://www.isca-archive.org/interspeech_2025/chen25d_interspeech.html']Target-timbre conversion with leakage control[/URL][/B] tackles this directly by replacing source-derived self-supervised features with closely matched target features before reconstruction. Pitch creates another constraint. Preserving the original F0 contour can keep the melody accurate, yet the source singer may operate in a range that does not suit the target voice. Some conversion systems compensate for pitch bias between singers instead of assuming every source note should be carried across unchanged. A male source pushed into a much higher target register, or the reverse, can expose the mismatch quickly. The model has to keep the musical line recognizable while making the result plausible for the target timbre. Pitch preservation can reduce target-singer similarity when the original register sits far outside the target voice's usual range. [HEADING=2]Real recordings expose failure modes that clean demos hide[/HEADING] A dry solo vocal gives the model a relatively clean source. A finished song is messier. Backing vocals, doubles, harmonies, reverb tails, distortion, and imperfect stem separation can all contaminate the features used for conversion. Harmony interference is especially troublesome because the system may detect more than one pitch where it expects a single lead line. Recent zero-shot systems have addressed F0 errors and harmony-contaminated inputs explicitly, including training with distorted or imperfect vocal stems. Without those protections, converted vocals can develop cracks, residual harmonies, unstable pitch, or blurred identity. Loudness and articulation create subtler carryover. A sharp attack, softened consonant, or swell into a long note can survive because those dynamics belong partly to the source performance. You can change who the vocal resembles without fully changing how the line was sung. Stem quality matters before conversion even begins. Harmony bleed or a poorly isolated double, can survive as unwanted pitch traces, even when the target timbre is convincing. Cleaner source separation reduces one class of error, but it cannot decide which expressive choices belong to identity and which belong to performance. [/QUOTE]

Name