Formant preservation in wide glissando (an octave and more)

[AvantGuy]

I want to take a long sustained prerecorded cello note on a single pitch, and bend the pitch at various pitch shift rates via automation. I'd like the freedom to do slow glissando, eg. bend an octave or more and also brief bends, say, < 1 sec. I'm thinking that an algorithm that does the long gliss properly would also work with brief bends. I believe I have to preserve formants, especially for bends that are less than a second or so with no sustain after target pitch is reached (looking for little "whoops" for some of these). There are other cases of doing long, slow bends which can sustain after the target pitch is reached).

I searched and of course found several threads on pitch shifts, ReaPitch, etc., but formants are often not mentioned or the bends are less than a semitone.
TIA very much!

[mschnell]

Obviously the instrument plugin you use is not prepared for that task.
-Michael

[AvantGuy]

Originally Posted by mschnell

Obviously the instrument plugin you use is not prepared for that task.
-Michael

Thanks Michael.

There's no plugin. The source sound is prerecorded cello samples so I'm looking for a transformation on an audio clip unless there's a better way. I wonder if there's a sampler that could manage this extreme pitch shift and preserve formants.

[jrk]

You can try ReaPitch, and with a suitable algorithm, you'll have the opportunity to adjust formants. Try automating the formant adjustment in the opposite direction to pitch.

[mschnell]

Originally Posted by AvantGuy

Thanks Michael.

There's no plugin. The source sound is prerecorded cello samples

I suppose it's close to impossible to create decently naturals sounding audio that way.
You might want to take a look at the Sample Modeling website to see what can be done by using a huge amount of samples and Kontakt scripting to create sample based virtual string instruments. E.g. their solo violin is nicely playable with my Roli Seaboard, and (really astonishingly) even can take Pitchbend of an octave width with rather natural sound.
-Michael

[jrk]

Not perfect, but formants up, pitch down:
https://sndup.net/3vqf

[Philbo King]

You might be able to do it using Reapitch, by making the formant control an envelope. Start with 0 Formant, end with -12 semitones, add points to tweak the curve of it to match the inverse of the pitch curve. Worth a try...

[Held]

Formants are specific frequencies related to vowels in speech. If I understand it correctly, the frequencies of the formants are fixed regardless of the pitch a person is singing in. So unless you want your cello to talk, they aren't really relevant to your task.

I think you might be thinking of timbre i.e. the relationship of overtones of your sound. The problem with normal frequency shifting is that these relationships get distorted and that's why pitch shifting was invented to keep the relationships intact.

In conclusion, any good quality pitch shifting plugin should do the trick.

Edit: But if you really need the formant preserving capabilities, take a look at https://www.meldaproduction.com/MHarmonizerMB

[jrk]

Originally Posted by Held

Formants are specific frequencies related to vowels in speech. If I understand it correctly, the frequencies of the formants are fixed regardless of the pitch a person is singing in. So unless you want your cello to talk, they aren't really relevant to your task.

That's a common meaning of "formant", but the concept isn't limited to speech. It makes perfect sense to refer to the contribution of body resonances (of a string instrument) as formants (which might be considered as a eq curve imposed on the basic sound). They'll be broadly constant regardless of the pitch of the note played. Obviously, simple pitch shifting will move these resonances - so that if I pitch shift a cello sound down, it'll sound unnatural in part because it's now sounding like a 5 foot "cello".

[EDIT] Here's a reference for you: https://newt.phys.unsw.edu.au/jw/formant.html

Some pitch shifters (including some of the algorithms supported by ReaPitch), allow you to move these "formants" so that in combination with the overall pitch shift you'll get a sound with a different pitch, but the formants in the "right" place.

[Held]

Originally Posted by jrk

It makes perfect sense to refer to the contribution of body resonances (of a string instrument) as formants (which might be considered as a eq curve imposed on the basic sound). They'll be broadly constant regardless of the pitch of the note played. Obviously, simple pitch shifting will move these resonances - so that if I pitch shift a cello sound down, it'll sound unnatural in part because it's now sounding like a 5 foot "cello".

[EDIT] Here's a reference for you: https://newt.phys.unsw.edu.au/jw/formant.html

That's cool. Thanks for the link.

Originally Posted by jrk

Some pitch shifters (including some of the algorithms supported by ReaPitch), allow you to move these "formants" so that in combination with the overall pitch shift you'll get a sound with a different pitch, but the formants in the "right" place.

Do you know how these formants are detected by plugins? Because they aren't necessarily the loudest partials, and they vary widely between instrumensts, right?

[jrk]

I am by no means an expert, but I gather it's less to do with "detecting" the formants and more to do with how the "grains" of the pitchshifting are manipulated.

[AvantGuy]

Originally Posted by jrk

You can try ReaPitch, and with a suitable algorithm, you'll have the opportunity to adjust formants. Try automating the formant adjustment in the opposite direction to pitch.

N00b alert: One can apply one's own algorithm to ReaPitch(?!) I must do more RTFM-ing! Appreciate this crucial tip, jrk.

[AvantGuy]

Originally Posted by jrk

Not perfect, but formants up, pitch down:
https://sndup.net/3vqf

Your downward gliss shows much promise. Thanks for tasking time to make this demo, jrk. I'm assuming your source sound was a recording of live cello that you directly treated as opposed to using a sampler like Kontakt (an idea for which was posited here as well).

Originally Posted by jrk

That's a common meaning...Here's a reference for you: https://newt.phys.unsw.edu.au/jw/formant.html

Some pitch shifters (including some of the algorithms supported by ReaPitch), allow you to move these "formants" so that in combination with the overall pitch shift you'll get a sound with a different pitch, but the formants in the "right" place.

Is it correct to assume one must notch out (or attenuate, in practice) the formant in the original BEFORE the pitch shift; then layer in the resonance sounds to reconstitute the formants?

Originally Posted by jrk

I am by no means an expert, but I gather it's less to do with "detecting" the formants and more to do with how the "grains" of the pitchshifting are manipulated.

One time I used ReaPitch to transpose vocals, although the shift was no more than a semitone. I thought it sounded fine and because I ran with whatever ReaPitch defaulted to I assumed that the algorithm it used did "find" the original formant in order to suppress it.

Now I'm wondering whether it ignored formants and in the range of <1 semitone it's simply not noticeable.
[edit] ...and there was no gliss-making involved.

[AvantGuy]

Originally Posted by mschnell

I suppose it's close to impossible to create decently naturals sounding audio that way.
You might want to take a look at the Sample Modeling website to see what can be done by using a huge amount of samples and Kontakt scripting to create sample based virtual string instruments. E.g. their solo violin is nicely playable with my Roli Seaboard, and (really astonishingly) even can take Pitchbend of an octave width with rather natural sound.
-Michael

Yeah, I had my doubts from the onset about artifacts that could creep in.

Oh that Sample Modeling vis Kontakt might give me solutions to things beyond this pitchshift/formant issue! I've been on the lookout for ways to virtualize extended techniques for the string family (eg. bowing the tailpiece, bowing the cello end pin, playing the strings behind the bridge, various col legno and sul ponticello effects, and preparations like inserting resonating objects between the strings, etc). I'm gonna be all over that site, thanks!

[AvantGuy]

Originally Posted by Philbo King

You might be able to do it using Reapitch, by making the formant control an envelope. Start with 0 Formant, end with -12 semitones, add points to tweak the curve of it to match the inverse of the pitch curve. Worth a try...

Thanks.
I'm a tryin'

[AvantGuy]

Originally Posted by Held

That's cool. Thanks for the link.

Do you know how these formants are detected by plugins? Because they aren't necessarily the loudest partials, and they vary widely between instrumensts, right?

Are these formants truly partials? I don't think so, but as for their amplitude, yes, I'd expect them to be in the quiet range similar to the amplitude at least of the instrument's lower partials.

[AvantGuy]

Originally Posted by AvantGuy

...then layer in the resonance sounds to reconstitute the formants?

"Layer in...", man, is that naive of me or what?! I thought about this and realize that the dynamics of the original performance have to dictate the dynamics of the "injected" formant. This would seem to at least involve envelope following of the source.

[mschnell]

Originally Posted by AvantGuy

Oh that Sample Modeling vis Kontakt might give me solutions to things beyond this pitchshift/formant issue! I've been on the lookout for ways to virtualize extended techniques for the string family (eg. bowing the tailpiece, bowing the cello end pin, playing the strings behind the bridge, various col legno and sul ponticello effects, and preparations like inserting resonating objects between the strings, etc). I'm gonna be all over that site, thanks!

For such even more extended modulation, sample based algorithms are not the way to go. Here "physical modeling" kicks in.

One of the leading technology here is "SWAM" by the "Audio Modeling" company. They have great string instruments (and Wind instruments).

-Michael

[AvantGuy]

Thanks Michael. I discovered SWAM during my research into SM. Looks interesting. However, I thought SM's solo strings sounded better than Audio Modeling. I didn't compare the other instruments (I wasn't very excited about SM's trumpet, TBH). A basic standard feature of strings, the mute, appears not even implemented in SM. Not sure about AM.

[mschnell]

In many cases (and obviously) the basic sound of sample based instruments seems more natural, while the handling of articulations is more extreme, versatile, and often more "natural" with physical modeling.

-Michael

[jrk]

Originally Posted by AvantGuy

N00b alert: One can apply one's own algorithm to ReaPitch(?!) I must do more RTFM-ing! Appreciate this crucial tip, jrk.

Errm, you can choose an algorithm (from the supplied) most of these afford formant "preservation" or adjustment.