Old 03-22-2019, 03:09 AM   #1
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 143
Default Making an icon bob by sidechain audio?

I wanted to do a little video where there's a small icon (a face) overlaid as a layer on top of a video, and make it shake and bob according to an audio input.

It's almost possible. Almost.

To make it key with the alpha map correctly you need to insert a dedicated video processor (otherwise the underlying video will get moved as well). However the parameters of this dedicated video processor cannot be modulated or automated.

Then the icon needs to be added as another media item (an alpha mapped .PNG image), with its own video processor (pan/zoom) as an item FX. These parameters can be automated but only on envelopes on the item itself. Modulation can be added: you can add an LFO, but the LFO's parameters cannot be automated, only set permanently. If you add an audio sidechain, it will not listen to the signal on the track (presumably because the item itself doesn't really receive anything, it's a sender).

So, is there currently any way to achieve that "little icon that bobs up and down according to audio, on top of underlying video" goal in Reaper?

Last edited by Skaven252; 03-22-2019 at 03:16 AM.
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 04:51 AM   #2
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
Default

Skaven252,

this should be possible but only with a workaround. Someone made a JSFX which allowed to translate audio to video parameter automation. I will see if I can find it.

Edit: Found it. Its called "audio to slider" and makes it possible to use a sidechain or main signal as an automated slider. You then can use this slider to automate the video parameter.



Search the forums for the plugin. Or compile it yourself. Here is the code for the audio to slider JSFX. (I hope this is allowed to post here as it is not my plugin.)

Code:
---------------------------------------------
--------------Audio to Slider----------------
------Outboarder   @2017  Reaper 5.50 -------
---------------------------------------------

slider1:0<0,1,0.000001>Left
slider2:0<0,1,0.000001>Right
slider3:M=3<0,10,0.01>Mul

@init
slider1 = 0;
slider2 = 0;
slider3 = 0;

@slider

@sample
    ampL = abs(spl0);
    ampR = abs(spl1);
    peakL = max(peakL, ampL);
    peakR = max(peakR, ampR);
    
    time_precise()-T > 0.02 ? ( 
       SL = 0;
       SR = 0;
       T=time_precise();
    );
    
    SL = max(SL, ampL);
    SR = max(SR, ampR);
   
    L = SL*M;  
    R = SR*M;
    
    slider1 = L;
    slider2 = R;
@block
Short overview of how to do it:
- Put you PNG file on a track
- Use the image overlay preset to make it transparent and proportional. (No dedicated VP needed)
- Put the background video on the track below the image
- Load the "audio to slider" JSFX before the video processor in the PNG FX chain
- Send audio to the PNG track but deselect the master send to avoid doubled volume
- The sliders of the "audio to slider" plugin should move now
- Go to the video processor image overlay preset, click once on the zoom parameter knob and go into the "param"-menu
- Choose Parameter modulation, MIDI link
- Open the drop-down menu and choose the left or right slider of the "audio to slider" FX
- Adjust the automation parameters (strength etc.) to fit to your expectations.

Hope that helps
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆

Last edited by Eliseat; 03-22-2019 at 05:16 AM.
Eliseat is offline   Reply With Quote
Old 03-22-2019, 04:57 AM   #3
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 143
Default

Quote:
Originally Posted by Eliseat View Post
Someone made a JSFX which allowed to translate audio to video parameter automation.
Is it the JS: video sample peeker? That's meant to work with the Oscilloscope and Spectrogram video processor presets, but it can't be used to automate or modulate any parameter. Are you referring to something else perhaps?

It's a matter of being unable to link things together. I tried a ReaControlMIDI on the track, but you can't link a FX that is inside an item, to a MIDI parameter that's in a track, there's no connection.

Sidechain modulation listening to track channel 1-2 doesn't receive any audio, if the effect parameter to be modulated is nested inside an item. Because the item doesn't receive any audio from a track, the traffic goes the other way around.

So it's a bit of a fundamental problem of not being able to link things together "the other way around".
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 05:22 AM   #4
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 143
Default

Awesome, thank you Eliseat!

So it exploits the workaround that parameters between different plugins in the same chain (or a different track) can be linked together. Clever!
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 05:45 AM   #5
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 143
Default

Quote:
Originally Posted by Eliseat View Post
- Load the "audio to slider" JSFX before the video processor in the PNG FX chain
- Send audio to the PNG track but deselect the master send to avoid doubled volume
- The sliders of the "audio to slider" plugin should move now
I can see in that preview GIF that it's doing what I wanted - but I seem to be running into the same problem as before:
- If I put the Image Overlay as a Track FX on the PNG track, it offsets the background video as well. But the AudioToSlider modulation works.
- If I put the AudioToSlider FX -> Image Overlay as Item FX into the PNG item, the AudioToSlider doesn't receive any audio (item cannot receive from track?), thus the sliders don't move. But the zoom/offset applies to the image only, so that part works.
- If the Image Overlay FX is in the Item FX, and the Audio To Slider is in the Track FX, the Image Overlay cannot link to the Audio To Slider plugin's parameter - it has to be in the same chain apparently.
- If I use a Dedicated video processor item, its parameters cannot be linked or automated at all (no such menus or buttons available in dedicated script items).

How was the result you see in that GIF done? Because it's reacting to the audio level, yet not zooming the background video, only the overlay. I thought I did everything in the order you instructed. [scratches head]

Last edited by Skaven252; 03-22-2019 at 07:09 AM.
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 08:01 AM   #6
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
Default

Uhm, for some reason the gif doesn't show an animation anymore. ¯\_(ツ)_/¯

Anyway. I did nothing special. If you use the latest image overlay it should allow to scale the PNG without affecting the background video. But if you want, I could recreate my project and send it to my file storage so you can test it on your system.

Are you using Mac or Windows? I'm on Windows.

Okay, I just make that same project and upload it somewhere. This should be the easiest way to show you how I made this.

Greetings
Eli
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-22-2019, 08:07 AM   #7
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 143
Default

I'm using the latest Reaper (5.973) so the Video Processor should be latest as well (it includes the video sample peeker and the oscilloscope + spectrogram video presets). On Windows 10. And I have the latest FFMpeg DLLs, if it depends on that.

Did you put the Image Overlay effect into the track chain, or the item chain?

Quote:
Okay, I just make that same project and upload it somewhere. This should be the easiest way to show you how I made this.
Awesome, thank you!
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 08:09 AM   #8
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 143
Default

All right, this is weird... instead of modifying an older project to those specifications, I created a new project from scratch. Now the Image Overlay as a Track FX does not affect the underlying video.

So... uh... it depends on in what order you do things? Hmmmmmmm.
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 08:19 AM   #9
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
Default

No, I guess it loaded the code of the old Image overlay preset which did not take care of the background video. So this would have been the next question of mine if your preset code looks the same as mine.

Anyway. The upload is nearly done. I will post the links soon. If you don't need it after finding out what went wrong, maybe other people can use it to learn.

Here is the link for the example project: https://mega.nz/#!vuwjWIwQ!mEp884DkG...fCxdpqhCI9MlLY

Have fun

Edit: For testing purposes, let me know if the project works.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆

Last edited by Eliseat; 03-22-2019 at 08:24 AM.
Eliseat is offline   Reply With Quote
Old 03-22-2019, 08:29 AM   #10
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
Default

By the way. Somewhere I asked Justin why we can't use audio as an modulation source for video parameters. He didn't really answer but told me it should not work. Maybe there was/is a limitation in doing so.
But I see no difference between using this workaround with a slider as the source vs. directly using audio.

As always in Reaper: You can do whatever you want if you know how to find a workaround.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-22-2019, 08:34 AM   #11
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 143
Default

Thank you!

Yes, your example project is working. Thanks! I had to replace and re-link the Param To Slider plug-in as I had it with a different file name on my end, but after that I could make the sphere zoom with the kick drum.

On my end, it does seem like there are some oddities and dependencies happening. If the audio track is multi-channeled (5.1 surround in my case), the Audio To Slider doesn't work via a send. I'll need to switch the channel mode of the audio file itself to stereo - just setting the track channels won't help. Even if the Send only sends channels 1-2. This is a bit odd because Audio To Slider itself is 4-channeled.

So, as long as you create the tracks in the right order and use stereo audio only, it works. ^_^
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 05:48 PM   #12
Bri1
Banned
 
Join Date: Dec 2016
Location: England
Posts: 2,432
Default

heh- take a peek at this lolz..>




Code:
desc:Disco-Lara
^if another js user would take this code justin wrote,and run with it--there could be a possibility of some better animations-monty python style..lolz.
just swap out the png's.. there's another with the bone guy=lolz..funny jsfx!
Bri1 is offline   Reply With Quote
Old 03-22-2019, 11:49 PM   #13
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
Default

Bri1 ...

where the heck did you find something like this? ლ(´ڡ`ლ)

Its kind of disturbing to see such a weird combination of music and graphic. But then I realize that it was you who posted it and everything is as it is supposed to be.

Skaven252,

there is another version of the "audio to slider" JSFX, maybe this can manage multi track. I don't know. For some reason I can't find the thread where this got posted anymore. So here is the code for the multi channel version.

Code:
------------------------------------------------------
------------Audio to Slider  4X(stereo)---------------
------------------------------------------------------
----------Outboarder   @2017  Reaper 5.50 ------------
------------------------------------------------------

slider1:0<0,1,0.000001>Left(1)
slider2:0<0,1,0.000001>Right(2)
slider3:M1=2<0,10,0.01>Mul A
slider4:Time1=0.02<0,0.5,0.001>Time A

slider5:0<0,1,0.000001>Left(3)
slider6:0<0,1,0.000001>Right(4)
slider7:M2=2<0,10,0.01>Mul B
slider8:Time2=0.02<0,0.5,0.001>Time B

slider9:0<0,1,0.000001>Left(5)
slider10:0<0,1,0.000001>Right(6)
slider11:M3=2<0,10,0.01>Mul C
slider12:Time3=0.02<0,0.5,0.001>Time C

slider13:0<0,1,0.000001>Left(7)
slider14:0<0,1,0.000001>Right(8)
slider15:M4=2<0,10,0.01>Mul D
slider16:Time4=0.02<0,0.5,0.001>Time D

@init
slider1 = 0;
slider2 = 0;
slider3 = 2;
slider4 = 0;
slider5 = 0;
slider6 = 0;
slider7 = 2;
slider8 = 0;
slider9 = 0;
slider10 = 0;
slider11 = 2;
slider12 = 0;
slider13 = 0;
slider14 = 0;
slider15 = 2;
slider16 = 0;
@slider


@sample
    ///////////////////////A (1/2)///////////////////////////
    ampL1 = abs(spl0);
    ampR1 = abs(spl1);
    peakL1 = max(peakL1, ampL1);
    peakR1 = max(peakR1, ampR1);
    
    time_precise()-T1 > Time1 ? ( 
       SL1 = 0;
       SR1 = 0;
       T1=time_precise();
    );
    
    SL1 = max(SL1, ampL1);
    SR1 = max(SR1, ampR1);
   
    L1 = SL1*M1;  
    R1 = SR1*M1;
    
    slider1 = L1;
    slider2 = R1;
    ////////////////////////B (3/4)/////////////////////////////
    ampL2 = abs(spl2);
    ampR2 = abs(spl3);
    peakL2 = max(peakL2, ampL2);
    peakR2 = max(peakR2, ampR2);
    
    time_precise()-T2 > Time2 ? ( 
       SL2 = 0;
       SR2 = 0;
       T2=time_precise();
    );
    
    SL2 = max(SL2, ampL2);
    SR2 = max(SR2, ampR2);
    
    L2 = SL2*M2;  
    R2 = SR2*M2;
    slider5 = L2;
    slider6 = R2;
    ////////////////////////C (4/5)/////////////////////////////
    ampL3 = abs(spl4);
    ampR3 = abs(spl5);
    peakL3 = max(peakL3, ampL3);
    peakR3 = max(peakR3, ampR3);
    
    time_precise()-T3 > Time3 ? ( 
       SL3 = 0;
       SR3 = 0;
       T3=time_precise();
    );
    
    SL3 = max(SL3, ampL3);
    SR3 = max(SR3, ampR3);
    
    L3 = SL3*M3;  
    R3 = SR3*M3;
    slider9 = L3;
    slider10 = R3;
    /////////////////////////D (6/7)////////////////////////////
    ampL4 = abs(spl6);
    ampR4 = abs(spl7);
    peakL4 = max(peakL4, ampL4);
    peakR4 = max(peakR4, ampR4);
    
    time_precise()-T4 > Time4 ? ( 
       SL4 = 0;
       SR4 = 0;
       T4=time_precise();
    );
    
    SL4 = max(SL4, ampL4);
    SR4 = max(SR4, ampR4);
    
    L4 = SL4*M4;  
    R4 = SR4*M4;
    slider13 = L4;
    slider14 = R4;
    /////////////////////////////////////////////////////
@block
I'm also not sure why you can't get the 2 channel version working with multi track audio. Seems a bit weird to me. I will take a look into it if I find a multi track video somewhere.

In general I would suggest to place a second version of the multi channel audio on a separate track by exploding the audio of the video. As I remember right you have to right click the multi track file and choosing something like "item processing"-> Explode to single tracks.
You than get copies of every channel which you could easily use with the audio to slider JSFX.

__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-23-2019, 01:52 AM   #14
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 143
Default

Thanks for the update Eliseat!

Now I'm getting ambitious. It should be possible to make multi-band bobbers by using a 3-Band Splitter, followed by 3 AudioToSliders on the bands, each bobbing a different image overlay layer.

... but there's no need to go more elaborate than that though, since beyond that you can just use the decorative spectrum analyzer video processor preset.
Skaven252 is offline   Reply With Quote
Old 03-23-2019, 03:53 AM   #15
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
Default

You already are ambitious.

The problem is, that you will quickly run into preview issues as this takes a lot of software computation. But I see you finally managed to get into the Reaper philosophy: Its not a DAW it a big audio (and video) tool set to play with.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-23-2019, 06:58 AM   #16
Bri1
Banned
 
Join Date: Dec 2016
Location: England
Posts: 2,432
Default

Quote:
Its kind of disturbing to see such a weird combination of music and graphic. But then I realize that it was you who posted it and everything is as it is supposed to be.

taking the piss m8 ?

problem is not i-it's others lack of sensing the messages that are given here-in plain english (90% of time it's completely legible) unless blinded by ignorance.. =)
cockos justin wrote 2 jsfx,to do with png animating..go hunting if it pleases...the potential is further reaching than half_ass attacks may be..

there is the underlying thing with vids in reaper being able to play on top,or below each track---the ordering of effects is a pita tbh..videos could mix,as audio does,but that's just not happening right now is it? ..devs doing some half_ass baking,or,overcook=crumbling product. imo.
Bri1 is offline   Reply With Quote
Old 03-23-2019, 09:12 AM   #17
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
Default

Quote:
Originally Posted by Bri1 View Post
taking the piss m8 ?

problem is not i-it's others lack of sensing the messages that are given here-in plain english (90% of time it's completely legible) unless blinded by ignorance.. =)
cockos justin wrote 2 jsfx,to do with png animating..go hunting if it pleases...the potential is further reaching than half_ass attacks may be..

there is the underlying thing with vids in reaper being able to play on top,or below each track---the ordering of effects is a pita tbh..videos could mix,as audio does,but that's just not happening right now is it? ..devs doing some half_ass baking,or,overcook=crumbling product. imo.
Bri, I don't understand your language. What is m8 and why do you write something like "taking the piss"? If this is a metaphor or picture, I don't get it. "Go hunting if it pleases"?
As a German its not easy to read everything here in English but your posts are even heavier to read as plain English. Sorry, this is no offend. Maybe I'm just to stupid to get your points.

And what I meant with "crazy" wasn't meant negative. As I just wanted to point out that no one else would have posted such a video here in this context. It seems everything you interested in is kind of abstract. That's fine. No insult. Its just obvious you. Nothing more.

Sorry, for all the misunderstandings.

If you are angry about the strange video handling aka - the effect of the very top video track affects all video tracks below, - then I can give you relief because since the last update some presets (like the image overlay we were talking here in this thread) really allow track dependent effects. Its a start. But a good one!

And I agree. The default video behavior seems kind of weird because one intuitively thinks everything works like audio. But as always, in most cases there is a workaround in Reaper.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-23-2019, 11:21 AM   #18
Bri1
Banned
 
Join Date: Dec 2016
Location: England
Posts: 2,432
Default

Quote:
As a German its not easy to read everything here in English......



If you are angry about.....

lolz-see m8,being a mate,friend buddy of iam=you. you must realize,this is an actual english typing forum.
there is a german forum i believe if your english is poorly developed...jus say'n bruv/sis.
+ you clearly have inserted this "anger" -when clearly,there is none =lol?
i encourage inventiveness+all sorts of expressions,but assumptions or unintentionally misleading ideas about a certain character,or a 1 sided perception of certain behaviours--is clearly,not in order..more 'out of order' =quite insulting when you look into the words,not @ them..


what's the topic again?

oh yeah icon bob by sidechain? did anyone here actually give a suitable response to that? nope?
hmmm.. could supply another reply to help further..but imo, it's best people work stuff out for themselves,in all honesty.. thanx.
Bri1 is offline   Reply With Quote
Old 03-23-2019, 11:38 AM   #19
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
Default

Do these "disco divas" belong to icon bob? Because I didn't find any valuable info about that. Where is the code of those presets? I just have no idea. (As always.)

And don't point me to the German forum, m8! <- please look at the cool British abbreviation someone taught me
I didn't mean that I don't understand English. Only yours seems to be encrypted. That's all.
So you suggest to look into the words instead of reading them? Uhm, do I need booze or drugs to find out what they mean or do I need more patience? Because this doesn't help at all! (☞゚∀゚)☞
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-23-2019, 12:15 PM   #20
Bri1
Banned
 
Join Date: Dec 2016
Location: England
Posts: 2,432
Default

Quote:
Do these "disco divas" belong to icon bob?
heh- yes,that's right m8,they belong to bob..try not to worry though,the nice young men with white jackets will be along to help,soon...soon.

someone else must know these older jsfx? i can't seem to remember the actual links,and think there are 2 folders for the 2 different js png files.

Code:
desc:Disco-Lara
desc:Disco-Slim
as i tried to suggest previously--if any js user could, (or more importantly) would, modify these 2 js- they could become little workshops in their own right= make them fully righteous+usable for all!! =please+ty.
Bri1 is offline   Reply With Quote
Old 03-26-2019, 01:12 AM   #21
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 143
Default

Quote:
Originally Posted by Bri1 View Post
oh yeah icon bob by sidechain? did anyone here actually give a suitable response to that? nope?
I used the wrong term in the subject line.

I was just looking for a way to bob an icon by audio, and had a problem with having to place the "Image Overlay" preset into the item FX instead of track FX to keep it from moving the background video; so the item FX couldn't listen to the audio from the send, because you can't send to an item (!), you can only send to a track.

"Sidechain" in here could mean that it's using a send to another channel, and that send won't be audible in the final output, it's only used to produce the values for the bobbing.
Skaven252 is offline   Reply With Quote
Old 03-23-2022, 04:32 AM   #22
Buy One
Human being with feelings
 
Buy One's Avatar
 
Join Date: Sep 2019
Posts: 1,131
Default

Quote:
Originally Posted by Bri1 View Post

someone else must know these older jsfx? i can't seem to remember the actual links,and think there are 2 folders for the 2 different js png files.

Code:
desc:Disco-Lara
desc:Disco-Slim
https://stash.reaper.fm/v/5302/Disco.zip

referenced at https://forum.cockos.com/showthread.php?p=1179311
__________________
https://github.com/Buy-One/REAPER-scripts (175)
REAPER is a DAW whose user guide file is larger than its installation file
Buy One is online now   Reply With Quote
Old 03-23-2022, 07:14 AM   #23
papagirafe
Human being with feelings
 
papagirafe's Avatar
 
Join Date: Aug 2020
Location: Brasil
Posts: 679
Default

Sorry folks for jumping late in the discussion, but from my experience parameter modulation for video positioning has never worked reliably (jumpy & erratic). I have tested positioning with success using the "gmem" mechanism between two video item presets. Never tried it from a JSFX to a video preset though but chances are good it would work.

Last edited by papagirafe; 03-23-2022 at 07:35 AM.
papagirafe is offline   Reply With Quote
Old 04-02-2022, 02:39 AM   #24
nirospring
Human being with feelings
 
Join Date: Aug 2011
Posts: 48
Default

You mentioned - "Compile it yourself." Is this easy to do? Can I copy and paste this somewhere to make a plugin?

Code:
---------------------------------------------
--------------Audio to Slider----------------
------Outboarder   @2017  Reaper 5.50 -------
---------------------------------------------

slider1:0<0,1,0.000001>Left
slider2:0<0,1,0.000001>Right
slider3:M=3<0,10,0.01>Mul

@init
slider1 = 0;
slider2 = 0;
slider3 = 0;

@slider

@sample
    ampL = abs(spl0);
    ampR = abs(spl1);
    peakL = max(peakL, ampL);
    peakR = max(peakR, ampR);
    
    time_precise()-T > 0.02 ? ( 
       SL = 0;
       SR = 0;
       T=time_precise();
    );
    
    SL = max(SL, ampL);
    SR = max(SR, ampR);
   
    L = SL*M;  
    R = SR*M;
    
    slider1 = L;
    slider2 = R;
@block
Short overview of how to do it:
- Put you PNG file on a track
- Use the image overlay preset to make it transparent and proportional. (No dedicated VP needed)
- Put the background video on the track below the image
- Load the "audio to slider" JSFX before the video processor in the PNG FX chain
- Send audio to the PNG track but deselect the master send to avoid doubled volume
- The sliders of the "audio to slider" plugin should move now
- Go to the video processor image overlay preset, click once on the zoom parameter knob and go into the "param"-menu
- Choose Parameter modulation, MIDI link
- Open the drop-down menu and choose the left or right slider of the "audio to slider" FX
- Adjust the automation parameters (strength etc.) to fit to your expectations.

Hope that helps
[/QUOTE]
nirospring is offline   Reply With Quote
Old 04-02-2022, 08:53 AM   #25
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,271
Default

Yeah even with something like the parameter linking described above, trying to modulate video from the audio thread doesn't work. You can do everything mentioned there and then write the video processor parameter as automation. Then disconnect the parameter linking and just run off the automation. Or for example you could use one of the dynamics>envelope scripts on your audio track, make that envelope into an automation item and copy it over to the VP parameter you want to modulate. But you won't be able to get a good preview, so it's a bit of a hit or miss, trial and error kind of thing.

The right way to do it is via the video peeker or something similar. I've done a few different things with that, and am working on some more. For what I think you're trying to do here, probably the best is my Audio Controlled Zoom/Rotate/Shift

But you might be able to get something out of Ash O Scope.

But you will find that driving these things straight from sample-by-sample (well, sample-by-frame or whatever...) will be pretty jumpy and weird because I have not added any smoothing or envelope detection in the VP preset itself. In the Zoom/Rotate... thread I detailed a relatively simple way to generate a smoother envelope with a JS plugin and PM which you can then send through sample peeker as a "Control Voltage" for the VP presets. It's essentially the opposite of the audio-to-slider plugin above, being slider-to-audio.
ashcat_lt is offline   Reply With Quote
Old 04-02-2022, 09:50 AM   #26
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,271
Default

Quote:
Originally Posted by nirospring View Post
You mentioned - "Compile it yourself." Is this easy to do? Can I copy and paste this somewhere to make a plugin?
Oh hey! Even though I kind of just said this isn't a great way to do it, I can help with making the JS. The easiest way is to open the FX Browser, right click in the left hand pane and find "Create new JS..." It will pop up a box where you can give it a file name (must be something your OS will accept as the name for a file) and description. Once you OK that, it'll add a new plugin to your track which has just some generic code in it. Hit the Edit button to open the JS editor. Click in that window and do Ctrl+A to select all the text. You can hit Backspace or Delete to clear it, or just leave it selected and it will be replaced. Copy the text from the code block in the above post. Paste it into the editor. While that JS editor is still in focus, hit Ctrl+S to save. That is now a JS plugin that you can find in your FX Browser whenever you want it.
ashcat_lt is offline   Reply With Quote
Old 04-03-2022, 11:00 PM   #27
nirospring
Human being with feelings
 
Join Date: Aug 2011
Posts: 48
Default

Awesome Ashcat! That's helpful! You give very detailed instructions. I'm getting a response, which is good.

Maybe this comes back to you saying that this isn't a great way to do this, but the Zoom is engaged to some extent as soon as the audio is detected. I'm modulating with a kick drum, so the zoom should only be a fraction of a second during the drum attack. Normally, with audio, you can set the attack, release, and minimum threshold so that the modulation is at rest, when you want it to be.

Should I keep playing around with it?




Quote:
Originally Posted by ashcat_lt View Post
Oh hey! Even though I kind of just said this isn't a great way to do it, I can help with making the JS. The easiest way is to open the FX Browser, right click in the left hand pane and find "Create new JS..." It will pop up a box where you can give it a file name (must be something your OS will accept as the name for a file) and description. Once you OK that, it'll add a new plugin to your track which has just some generic code in it. Hit the Edit button to open the JS editor. Click in that window and do Ctrl+A to select all the text. You can hit Backspace or Delete to clear it, or just leave it selected and it will be replaced. Copy the text from the code block in the above post. Paste it into the editor. While that JS editor is still in focus, hit Ctrl+S to save. That is now a JS plugin that you can find in your FX Browser whenever you want it.
nirospring is offline   Reply With Quote
Old 04-04-2022, 01:22 AM   #28
nirospring
Human being with feelings
 
Join Date: Aug 2011
Posts: 48
Default

When I try to put it to something less busy - in this case the snare - it just shakes constantly- even when there's no snare hits for long periods.





Quote:
Originally Posted by nirospring View Post
Awesome Ashcat! That's helpful! You give very detailed instructions. I'm getting a response, which is good.

Maybe this comes back to you saying that this isn't a great way to do this, but the Zoom is engaged to some extent as soon as the audio is detected. I'm modulating with a kick drum, so the zoom should only be a fraction of a second during the drum attack. Normally, with audio, you can set the attack, release, and minimum threshold so that the modulation is at rest, when you want it to be.

Should I keep playing around with it?
nirospring is offline   Reply With Quote
Old 04-04-2022, 03:20 AM   #29
papagirafe
Human being with feelings
 
papagirafe's Avatar
 
Join Date: Aug 2020
Location: Brasil
Posts: 679
Default

Quote:
Originally Posted by nirospring View Post
When I try to put it to something less busy - in this case the snare - it just shakes constantly- even when there's no snare hits for long periods.
There is probably some inaudible noise in the snare track. Try adding a gate fx at the beginning of the fx chain.
papagirafe is offline   Reply With Quote
Old 04-04-2022, 05:32 AM   #30
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,271
Default

If you’re doing it via PM, there’s attack and release in there. There’s also audio threshold settings which can act like a gate and limiter. There’s also a Bézier curve which both shows where your audio is hitting and allows you to curve the response quite a bit. You’ll still probably get some flickering or other weirdness until/unless you actually write the automation and disconnect the PM as I mentioned in my first reply above.
ashcat_lt is offline   Reply With Quote
Old 04-04-2022, 11:43 PM   #31
nirospring
Human being with feelings
 
Join Date: Aug 2011
Posts: 48
Default

I Youtubed in order to figure out how to write Parameter Modulation to Track Automation and the Youtube video (The Reaper Blog) mentioned Ashcat's name. Small world! Then I got sidetracked watching your (Ashcat's) Youtubes for a bit. I want to know where you live Ashcat. It seems like a place from the future and the past.

I tried gating on the snare track itself and no improvement. Ashcat - I'm not seeing any attack or release on the parameter modulation. Normally I do when selecting "Audio Control Signal (Sidechain)". But in order to connect to the Audio Slider I have to choose -"Link from MIDI of FX Parameter". Then I only see an option for "Offset" and Scale".




Quote:
Originally Posted by ashcat_lt View Post
If you’re doing it via PM, there’s attack and release in there. There’s also audio threshold settings which can act like a gate and limiter. There’s also a Bézier curve which both shows where your audio is hitting and allows you to curve the response quite a bit. You’ll still probably get some flickering or other weirdness until/unless you actually write the automation and disconnect the PM as I mentioned in my first reply above.
nirospring is offline   Reply With Quote
Old 04-05-2022, 08:37 AM   #32
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,271
Default

Quote:
Ashcat - I'm not seeing any attack or release on the parameter modulation. Normally I do when selecting "Audio Control Signal (Sidechain)". But…
Oh shit! Yeah sorry about that. I’ll use the excuse that it was very early and I’m a dumbass. The times I’ve done, I just used any arbitrary audio plugin, chose any of its parameters, and used Parameter Modulation to make that follow the audio, which I then linked to the VP. Then you get the all the A/R and other stuff. Pretty sure I tried something like Audio_to_Slider, but it didn’t help the flickering issue I was having.
Quote:
Then I got sidetracked watching your (Ashcat's) Youtubes for a bit…
Boy, I’m even more sorry about that!
Quote:
I want to know where you live Ashcat. It seems like a place from the future and the past.
Lol. My father calls it the Land Beyond Reality. I call my home/studio Five Miles From Nowhere because the nearest town is really nowhere, and I’m 5 miles from the city limits.
ashcat_lt is offline   Reply With Quote
Old 04-06-2022, 02:16 AM   #33
nirospring
Human being with feelings
 
Join Date: Aug 2011
Posts: 48
Default

It's all good. I finally got the result I wanted! I think maybe you were suggesting something like this, but I wasn't quite picking it up.

-I used ReaEQ Paramater Modulation to write Automation on the snare track. I basically pretended like it was the Zoom feature that I was trying to control and estimated how much I wanted it to move.

-I went to the Video Track and selected the Video Processor plugin. Then from the plugin I selected "show track envelope" which put Zoom (last touched) into view

-I copied all points in the ReqEQ automation over to the Video Processor Zoom automation

-There was a delay in the video effect from the sound. So I grabbed the entire automation and manually moved it over. Since the delay was consistent, this got everything perfectly synced.

It's not a perfect process, but it beats the hell out of manually doing each automation - which I was trying to do with Adobe Premiere.

Quote:
Originally Posted by ashcat_lt View Post
Oh shit! Yeah sorry about that. I’ll use the excuse that it was very early and I’m a dumbass. The times I’ve done, I just used any arbitrary audio plugin, chose any of its parameters, and used Parameter Modulation to make that follow the audio, which I then linked to the VP. Then you get the all the A/R and other stuff. Pretty sure I tried something like Audio_to_Slider, but it didn’t help the flickering issue I was having.

Nah, it was fun- definitely different vibe and interesting. It's really weird to think about how casually you interact with someone online not thinking much about how very different the other person's surroundings might be than your own.
Quote:
Originally Posted by ashcat_lt View Post
Boy, I’m even more sorry about that!

Are you in the states? Which one? I would love to live out in the middle of nowhere
where it would be possible to own your land/home and be somewhat self sustaining. I live in Southern California where everyone is working their asses off just to make rent.

Quote:
Originally Posted by ashcat_lt View Post
Lol. My father calls it the Land Beyond Reality. I call my home/studio Five Miles From Nowhere because the nearest town is really nowhere, and I’m 5 miles from the city limits.
nirospring is offline   Reply With Quote
Old 04-24-2022, 01:17 PM   #34
gunny
Human being with feelings
 
gunny's Avatar
 
Join Date: May 2014
Location: Ostrava, Czechia
Posts: 72
Default

Quote:
Originally Posted by ashcat_lt View Post
You’ll still probably get some flickering or other weirdness until/unless you actually write the automation and disconnect the PM as I mentioned in my first reply above.
I experimented with it a bit and I find it rather strange that the flickering affects even the "background video layer". (I.e. I've got a static background and a "sprite" with alpha channel which changes its opacity.)

Do you know what exactly goes wrong? It seems like some weird value from the PM sometimes disables the alpha channel of the sprite...?
gunny is offline   Reply With Quote
Old 04-24-2022, 02:01 PM   #35
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,271
Default

Quote:
Originally Posted by nirospring View Post
Are you in the states? Which one?
Technically I’m in WI, but when I want to sound cool, I say I’m from Duluth. It’s not like some homestead farmland. I call it “subrural” cause it’s not real suburbs, but it ain’t deep country either. We have a well, but otherwise are not particularly self-sufficient, and I still have a day job.
Quote:
Originally Posted by gummy;
Do you know what exactly goes wrong?
I guess I can’t really account for any specific behavior, but just in general the video thread is completely independent and asynchronous from the audio thread, and I guess it just sometimes looks for values when the audio thread isn’t putting them out or more likely the value the audio thread is putting out doesn’t actually match the time that the video processor thinks it’s processing, so things go weird.

The video peeker plugin gets around this by filling a rather long buffer with audio data and then also passing a pointer so that when the processor does go to update, it can pull the sample from where the peeker thinks it was. Ultimately the best answer is to hack that video peeker functionality into whatever preset you’re trying to use. It’s not generally all that complicated, but it does require custom code for each preset.
ashcat_lt is offline   Reply With Quote
Old 04-25-2022, 01:35 AM   #36
gunny
Human being with feelings
 
gunny's Avatar
 
Join Date: May 2014
Location: Ostrava, Czechia
Posts: 72
Default

Quote:
Originally Posted by ashcat_lt View Post
Ultimately the best answer is to hack that video peeker functionality into whatever preset you’re trying to use. It’s not generally all that complicated, but it does require custom code for each preset.
Unfortunately my knowledge of Reaper scripting is very limited so eventually I used hand-tuned envelope for now but I'd like to get back to it later.

I am not really sure what module you mean by "the video peeker".
Do you mean that incorporating the code from "Audio-to-Slider.js" directly into "Video processor: Image Overlay" would work better?
gunny is offline   Reply With Quote
Old 04-25-2022, 03:37 AM   #37
papagirafe
Human being with feelings
 
papagirafe's Avatar
 
Join Date: Aug 2020
Location: Brasil
Posts: 679
Default

Quote:
Originally Posted by gunny View Post
Unfortunately my knowledge of Reaper scripting is very limited so eventually I used hand-tuned envelope for now but I'd like to get back to it later.

I am not really sure what module you mean by "the video peeker".
Do you mean that incorporating the code from "Audio-to-Slider.js" directly into "Video processor: Image Overlay" would work better?
Video peeker is a JSFX module that can exchange info with 2 of the video processor video presets: "decorative..." . JSFX modules are sort of equivalent to a VST module and are meant for sound transformation. Video processor scripts are video code saved in preset space with each instance of "video processor fx". JSFXs and Video processor presets can share info through named global variables known as GMEMs. The two worlds (JSFX vs VP) are using different clocks so you need to synchronize them. The video peeker and the decorative presets are examples of that.

BTW I am assuming that the problem you are reporting occurs *after* rendering, right?
papagirafe is offline   Reply With Quote
Old 04-25-2022, 04:52 AM   #38
gunny
Human being with feelings
 
gunny's Avatar
 
Join Date: May 2014
Location: Ostrava, Czechia
Posts: 72
Default

Quote:
Originally Posted by papagirafe View Post
Video peeker is a JSFX module that can exchange info with 2 of the video processor video presets: "decorative..."
Um, now I see "JS: video sample peeker" is actually a standard Reaper JSFX.
I thought ashcat_lt was referring to some other module by loosely describing its function. =}

Quote:
Originally Posted by papagirafe View Post
BTW I am assuming that the problem you are reporting occurs *after* rendering, right?
Yes. It looks quite fine in Video Window within Reaper. It's not properly synchronized all the time but it does not flicker. The rendered video is synchronized but there are many black frames in it too.
gunny is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 03:25 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.