03-24-2020, 04:01 PM | #1 |
Human being with feelings
Join Date: Feb 2020
Location: San Diego, CA
Posts: 19
|
How to Sync Spatial Audio & 360 Video — Includes Reaper Project Download
We created a tutorial over on the Voyage Audio blog to help people sync Spatial Audio to 360 Video using Facebook 360 Spatial Workstation. There is also a downloadable Reaper Session to help with understanding the workflow. Let me know if we missed something and hope it can help some people out!
https://voyage.audio/how-to-sync-spa...-360-vr-video/
__________________
Download The Free Spatial Mic Reaper Session: https://voyage.audio/listen-to-spatial-mic/ |
10-15-2020, 09:12 AM | #2 |
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Nice tutorial.
I make an XML to Reaper conversion tool. It occurred to me that Final Cut Pro X which has native 360 video support also exports pan, tilt, roll metadata for each clip in it's XML exports. Currently my tool ignores this but it would be possible to convert this metadata into automation in reaper (especially easy if using the ATK as those are JS FX). Would this be useful? |
10-19-2020, 01:40 AM | #3 | |
Human being with feelings
Join Date: Oct 2019
Posts: 2
|
Quote:
Currently it is a complete time waster trying to re position spatial audio again, after the video has been rotated.. The perspective is always changed in the video edit, and all my intricate spatial locking goes to waste. Afterwards i have to automate by eye, ear, and SPL power map.If i am lucky (next to never) I have access to the NLE or the editor tells me what they have done. Then i have idea where to start. Either way it takes ages, and there is no automated process to do this in any DAW/NLE. I searched believe me 🙃 You would get loads of the spatial audio community on board with this function alone! If this could be done for Premier too, you would get them all! 😉 I use higher order plugins, not the atk, but any automation can be reassigned in reaper anyway. Even easier if its labelled... This is even useful for vbap/dbap plugs that allow rotation too.. Which i sometimes use for discreet channel based installations. Quite excited at the prospect! Cheers |
|
10-19-2020, 04:15 AM | #4 | |
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Worth considering that the 360 metadata belongs to the video item, not tracks or audio items. So how best to transfer it? Vordio already allows you to include video clips as empty items (it can be useful to see where the picture cuts are for sound design). So I could put a JSFX on the empty video item to hold the automation. How easy is it to move item FX automation to another item or track? Or else could dump the metadata into item notes and have a script transfer it. Vordio already dumps useful metadata such as scene/take into item notes. Another possibility would be to dump ADM axml (audio definition model) metadata. There is now a free reaper extension for importing ADM for spatial audio. https://www.bbc.co.uk/rd/blog/2020-1...software-tools https://www.youtube.com/watch?v=u7P5...ture=emb_title What will be the best workflow? Last edited by mrlimbic; 10-19-2020 at 04:23 AM. |
|
10-19-2020, 05:30 AM | #5 |
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Just checked Premiere XML. It doesn't include the VR rotate filter metadata in the XML.
"Translation issue: Sequence <wells ambi weds> at , video track 1: Effect <VR Rotate Sphere> on Clip <wells ambi weds.mov> not translated." However it may be possible to script Premiere to export axml for VR clips which could then be loaded in reaper using the EAR Production Suite. https://premiere-scripting-guide.rea...nentparam.html |
10-19-2020, 10:09 AM | #6 |
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
OK. I have got something basic working with FCPX.
These time=angle pairs get added to item notes for empty video items. The negative time values are not a bug. My test had out of range keyframes on clips in FCPX. I could filter them out but probably not necessary, especially if they are to be converted to track automation. A script could read these and apply them to plugin automation parameters. Add the the item position to parameter time and you'd have track automation point time. Static rotate (clip with no keyframes) Code:
TILT: 72.2522 PAN: -98.1707 ROLL: -267.598 Code:
TILT: -62.920=14.192, -50.760=14.192, 12.840=-254.793, 29.160=88.8078, 92.360=14.192 PAN: -62.920=-198.393, -50.760=-184.62, 12.840=-182.885, 29.160=-104.839, 92.360=-10.1287 ROLL: -62.920=0.0, -50.760=0.0, 12.840=177.096, 29.160=438.698, 92.360=0.0 Last edited by mrlimbic; 10-19-2020 at 10:15 AM. |
10-20-2020, 07:44 AM | #7 | |
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
However, that info is in the .pproj file. I could start supporting .pproj format in future but it is a lot of work. It's complex and undocumented so might take a long time to debug. So won't happen soon. However for FCPX I've done the first part already. Now I just need to write a reaper script that creates automation from the item notes. So that will be possible very soon. |
|
10-26-2020, 05:33 AM | #8 | |
Human being with feelings
Join Date: Oct 2019
Posts: 2
|
Quote:
If all good I'll spray some promo over the spatial audio forums. There will be a lot if happy people out there 😉 |
|
10-27-2020, 01:08 PM | #9 | |
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
It searches for parameter names, so in theory it should work with any plugins with roll, tilt, pan, yaw, pitch etc. However when I tried it with FB360 the VST3 will only accept hard min/max values, nothing in between. So 180 or -180 degrees but not 0. I don't know if this is a REAPER issue or a FB workstation VST issue. I'm very confused. Does the FB360 VST allow automation injection? Have you tried using other scripts that generate automation with it? |
|
10-28-2020, 06:59 AM | #10 | |
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
One limitation though is that FCPX allows infinite rotation (outside -180 to +180 range) but FB360 VST3 doesn't. So it won't work with large adjustments. The video editor will have to be aware of this & stick to -180 to +180. |
|
10-28-2020, 07:51 AM | #11 |
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
OK. I now have a working demo.
|
10-29-2020, 10:48 AM | #12 |
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
I now have an official release of the FCPX/FB360 VR rotate helper script in my mrlimbic reapack repository called "mrlimbic_apply vr rotate from item notes.lua"
https://github.com/mrlimbic/reascrip...ster/index.xml To test it you will need build Vordio 6.0.1747 or greater. The unlicensed version allows experimenting with short timelines (up to 3 mins). Downloads are here currently (this location is temporary and will change when new website for Vordio 6.0 is ready).. https://drive.google.com/drive/folde...93?usp=sharing You need to turn on the option called "include video as empty items." This feature will create empty items for video clips that contain metadata that the script uses. After converting the .fcpxml you can then open the RPP and use the script this way: 1. Select the empty item with the VR metadata in item notes 2. Select the FB360 track FX that should have the metadata applied to it's tilt, roll, pan parameters. 3. Run script This is first version and only works with track FX currently (item FX can come later). I've only tested it with the FB360 plugin. In theory it should work with any plugin that names parameters as tilt/pitch, roll, pan/yaw because it searches for words in the parameter names. |
Thread Tools | |
Display Modes | |
|
|