Surround up/downmixing questions

[raiszig]

Hello,

it's my first post but I've been reading and learning a lot in the past months here already, thanks to everybody for sharing their knowledge!
I'm coming from a film editing background, regularly working with a sound engineer in the final stages of projects and he motivated me to have a look at reaper some time ago to better prepare the audio for our studio sessions.
I've done very basic sound editing before, but Reaper is the first DAW I am really trying to learn from scratch and I enjoy it a lot!
There's a bunch of questions which popped up over the time in the learning process, one of them is this:

We are usually finishing our film/video projects with a stereo and a 5.1 surround mix (maybe expanding to atmos some day).
For this, we usually make a specific 5.1 mix in the studio and derive a stereo downmix for use with streaming services etc..
The thing I've never fully understood is: how can a simple downmix treat the surround information properly without producing any phasing issues (something I was told to avoid at all costs at the beginning of my sound-learning-path). By "simple downmix" I mean not using a plugin, but folding the sound within common given specs of routing and certain db attenuation for the surround channels.
As for plugins, I understand that commonly used plugins like Penteo or Nugen promise to take care of such phasing issues by detecting and eliminating them which obviously is a more complex under-the-hood process compared the simple fold down.
On their website Nugen states: "And that’s a big part of why we included delay compensation within Halo Downmix, in order to allow engineers to create better downmixes, without the phase cancellation caused by nasty artificial chorus and reverb involved in some legacy upmixing practices."
so I guess my question boils down to what is considered as "legacy upmixing techniques" and why have they been used before (since even mono compatibility must've been a thing already even in these "legacy" times?!). I guess it has to do with adding different reverbs in the L R nad Ls and Rs channels? And how would one go about this correctly to avoid phasing even with a "simple" fold down?

As for upmixing: I don't get how upmixing an existing 2-channel stereo mix of a film to let's say 7.1.4 (to take an extreme example) works, as surely neither Halo nor Nugen can dramaturgically analyze how a sound in the stereo mix is supposed to move i.e. front to back or where it is to be positioned in the overheads).
Or can such isolation of sounds in the upmix anyway only work for mono vs. stereo information, like a reverse mid side decoding where the plugin analyzes which signals are mono (like dialog) and therefore puts them in the center channel?
So I guess my question would be: is upmixing with plugins like penteo or nugen so widely used just for its massive time saving efficiency compared to re-mixing or are there any creative or technical advantages to this as well?

sorry for this long first post and many greets.

[serr]

Originally Posted by raiszig

As for upmixing: I don't get how upmixing an existing 2-channel stereo mix of a film to let's say 7.1.4 (to take an extreme example) works...

Anywhere from a sort of "happy accident" that sounds interesting even though not intentional to absolutely appalling IMHO.

There ARE some abilities to separate audio elements with spectral editing and AI learning. Excellent efforts take a LOT of fiddly work and time. The karaoke aimed apps go straight to lo-fi. Artifacty like low bit rate mp3.

Sometimes you do what you have to, right? It strays from intentional mixing but some listeners get excited about sounds coming from multiple speakers even if it doesn't make sense artistically. As a listener I'm interested in the sound of the intended mix for good or bad. Upmixing gets in the way of hearing the source it came from in it's original form. Also I'm thinking purely of music focused listening. Movies aren't so critical on the audio (unfortunately).

[mat chat]

Hi , so several methods are there to downmix a 5.1 to stereo.
If you just do a LoRo downmix ( L = L + C-3dB + Ls / R = R = C-3dB + RS), no phase issues are gonna be encountered but you won't be able to reconstruct the surround signal with an upmixer.
If you are using a LtRt method ( where you put on L (Ls + Rs) at a 180° phase shift) then you could have some minor issues
To reduce drastically these issues dolby recommand to use a 90° phase shift on Lt and a minus 90° phase shift on the Rt (with some coefficient)
The amount of Ls and Rs you send to the Lt and Rt are lower in volume at the first place and are attenuated by some coefficient, so that plus the 90° phase shift you almost never have a problem of phase. and you are able to reconstruct S , a surround signal for a later upmix.
more details here
you can "easily" do the matrix with Jesusonic mapper downmixer. I made a video on how to do it here but it's in french (it can maybe help you anyway).
Cheers.

[mat chat]

For upmixing, the reconstruction of the center signal deals with signal correlation and I didn't get to do such operation with the signal with reaper only.

[Lynx_TWO]

I don’t have a lot of experience with downmixing from 5.1 to stereo, but I’d probably stick an instance of Pi by SoundRadix on each channel before mixing down since the plugin will automatically rotate phase real time to avoid phase cancellation…

The downside might be that you’d lose the upmix functionality of a the resultant stereo mix, but that would be an interesting experiment…

[DVDdoug]

The thing I've never fully understood is: how can a simple downmix treat the surround information properly without producing any phasing issues (something I was told to avoid at all costs at the beginning of my sound-learning-path).

Phase issues ONLY happen when an exact copy is delayed or phase-shifted and then mixed with the original at about the same volume. That usually doesn't happen with an original surround mix from a multitrack source but it can happen with "artificial" upmixing or with certain effects. (Mono compatibility is a fairly-common problem with "fake stereo", but those recordings are rare and mono playbsck systems are rare except for the speaker in a smart phone.)

And, the mixing engineer should check the stereo downmix.


It COULD happen with a "Pro Logic" track, since Pro Logic encoding/decoding depends on phase differences, but Pro Logic mixes are done from the beginning to be stereo compatible so again, the mixing engineer should be checking the stereo playback.

Almost all of the 5.1 channel DVDs I have include an optional stereo track that was made as a separate stereo mix so I rarely listen to a real-time downmix.

I guess my question boils down to what is considered as "legacy upmixing techniques"

I don't know... I usually let my AVR upmmix at playback time. With Pro-Logic encoded movies, of course I use standard Pro-Logic decoding. Otherwise I leave them as stereo.

With stereo music I like to use a "soundfield" setting to get some delayed reverb in the rear channels.

As for upmixing: I don't get how upmixing an existing 2-channel stereo mix of a film to let's say 7.1.4 (to take an extreme example) works, as surely neither Halo nor Nugen can dramaturgically analyze how a sound in the stereo mix is supposed to move i.e. front to back or where it is to be positioned in the overheads).

I don't know how the pros do it... It seems like it's rarely done on commercial releases. I think all of the older movies I have only have the original mono or stereo soundtrack.

I made a 5.1 track from a mono concert recording once (or maybe twice). This was just for fun and I made no attempt to make it mono or stereo compatible but I kept the original mono with a menu option on the DVD.

For the music I used different-complementary EQ in the front channels to spread-out the sound for a "spacial feel". I didn't try to put the vocals in the center or to correctly position the other instruments.

I did pretty-much the same thing to get an "artificial stereo" mix in the rear. Then I added delay and reverb to the rear and lowered the level a bit compared to the front.

For the talking between songs I panned everything to the center (probably with a little reverb in the rear, but I don't remember).

For the applause I "stole" applause from different parts of the show and put it in the rear so it wasn't simply a modified copy of the front. And I "panned" the applause to be more dominate in the rear.

[serr]

There are encoded formats made to store surround mixes in 2 channel containers. A couple more modern lossless versions and a slew of lossy options. The lossless options decode to the original audio and will null 100% with it. The lossy options get more corrupt the lower quality you go, but they do decode to at least an image of the original.

There's no way to restore a surround mix from a stereo downmix of said mix! Downmixing isn't encoding. It's reducing the mix to stereo so it can at least be heard in stereo without grossly missing entire channels of audio. An intentional compromise to allow someone to preview a surround mix on only a stereo array. It will not "decode" back to the original surround anymore than you can get that egg back from a baked cake.

[raiszig]

wow, very interesting responses, thanks to everyone for chipping in with their thoughts and informations!!
I have to admit that for some of it, it will take me some time to wrap my head around, as some of it's close to the limit of my current technical knowledge in the audio sphere, but I'm very much motivated to learn and some things already became a lot more clear by all your informative replies!

Thanks @mat chat for the link to the dolby document on the behaviour of their renderer and also for the link to the video on the multichannel mapper, looks great, je le vais essayer a comprendre

@serr very interesting to learn also about philosophy on how to correctly treat a certain mix, while researching I also found another interesting comment of yours dating back to 2013 elaborating further on this related to surround / stereo mixes.

To the others, thanks a lot for your further elaborations on strategies and norms, also on the role of phase shift and attenuation of some of the downmixed channels and the associated effects.
Especially willful phase shift in such (or other related phasing issue) scenarios is something I understand to be helpful, but I have yet to completely grasp technically how such phase shifted signals interact with each other to resolve such issues. luckily there is a lot of information on phase shift online I gather, and still a lot of time to learn

[Lynx_TWO]

Originally Posted by serr

There's no way to restore a surround mix from a stereo downmix of said mix! Downmixing isn't encoding. It's reducing the mix to stereo so it can at least be heard in stereo without grossly missing entire channels of audio. An intentional compromise to allow someone to preview a surround mix on only a stereo array. It will not "decode" back to the original surround anymore than you can get that egg back from a baked cake.

Well, there you go then. Stick Pi from SoundRadix on each channel and downmix it to stereo in the prescribed format, which I believe (but could totally be wrong) is Center and Surround channels at -3dB, and LFE at +10 dB (since LFE is at -10dB originally for all the amplifiers that boost LFE by 10dB)

EDIT: found a better coefficient for surround to Stereo depending on channel count:

5.1 Surround downmix to stereo

FL, FC, FR, BL, BR, LFE -> FL, FR

FL = 0.374107*FC + 0.529067*FL + 0.458186*BL + 0.264534*BR + 0.374107*LFE
|
FR = 0.374107*FC + 0.529067*FR + 0.458186*BR + 0.264534*BL + 0.374107*LFE


7.1 Surround

FL, FC, FR, SL, SR, BL, BR, LFE -> FL, FR

FL = 0.274804*FC + 0.388631*FL + 0.336565*SL + 0.194316*SR + 0.336565*BL + 0.194316*BR + 0.274804*LFE
|
FR = 0.274804*FC + 0.388631*FR + 0.336565*SR + 0.194316*SL + 0.336565*BR + 0.194316*BL + 0.274804*LFE


6.1 Surround

FL, FC, FR, SL, SR, BC, LFE -> FL, FR

FL = 0.321953*FC + 0.455310*FL + 0.394310*SL + 0.227655*SR + 278819*BC + 0.321953*LFE
|
FR = 0.321953*FC + 0.455310*FR + 0.394310*SR + 0.227655*SL + 278819*BC + 0.321953*LFE


5.0 Surround

FL, FC, FR, BL, BR -> FL, FR

FL = 0.460186*FC + 0.650802*FL + 0.563611*BL + 0.325401*BR
|
FR = 0.460186*FC + 0.650802*FR + 0.563611*BR + 0.325401*BL


Quadraphonic Channel

FL, FR, BL, BR -> FL, FR

FL = 0.422650*FL + 0.366025*BL + 0.211325*BR
|
FR = 0.422650*FR + 0.366025*BR + 0.211325*BL


Linear Surround Channel

FL, FC, FR -> FL, FR

FL = 0.414214*FC + 0.585786*FL
|
FR = 0.414214*FC + 0.585786*FR

Just need to do some basic math to figure out what these values are in dB reduction; which I can do once I get home.

SOURCE: https://superuser.com/questions/8524...o-using-ffmpeg

EDIT: here you go; dB values:

dB Values:

5.1 Surround downmix to stereo

FL, FC, FR, BL, BR, LFE -> FL, FR

FL = -4.27dB FC + -2.7649dB FL + -3.3896dB BL + -5.7752dB BR + -4.27dB LFE
|
FR = -4.27dB FC + -2.7649dB FR + -3.3896dB BR + -5.7752dB BL + -4.27dB LFE


7.1 Surround

FL, FC, FR, SL, SR, BL, BR, LFE -> FL, FR

FL = -5.6098dB FC + -4.1046dB FL + -4.7293dB SL + -7.1149dB SR + -4.7293dB BL + -7.1149dB BR + -5.6098dB LFE
|
FR = -5.6098dB FC + -4.1046dB FR + -4.7293dB SR + -7.1149dB SL + -4.7293dB BR + -7.1149dB BL + -5.6098dB LFE


6.1 Surround

FL, FC, FR, SL, SR, BC, LFE -> FL, FR

FL = -4.9221dB FC + -3.4169dB FL + -4.0416dB SL + -6.4272dB SR + -5.5468dB BC + -4.9221dB LFE
|
FR = -4.9221dB FC + -3.4169dB FR + -4.0416dB SR + -6.4272dB SL + -5.5468dB BC + -4.9221dB LFE


5.0 Surround

FL, FC, FR, BL, BR -> FL, FR

FL = -3.3707dB FC + -1.8655dB FL + -2.4902dB BL + -4.8758dB BR
|
FR = -3.3707dB FC + -1.8655dB FR + -2.4902dB BR + -4.8758dB BL


Quadraphonic Channel

FL, FR, BL, BR -> FL, FR

FL = -3.7402dB FL + -4.3649dB BL + -6.7505dB BR
|
FR = -3.7402dB FR + -4.3649dB BR + -6.7505dB BL


Linear Surround Channel

FL, FC, FR -> FL, FR

FL = -3.8278dB FC + -2.3226dB FL
|
FR = -3.8278dB FC + -2.3226dB FR

[mat chat]

Originally Posted by raiszig

je le vais essayer a comprendre