03-13-2021, 04:36 AM | #1 |
Human being with feelings
Join Date: Feb 2021
Posts: 11
|
Audio visualizer for video
Is it possible and how to create waveaudio visualizer like in file attached? Thanks
|
03-13-2021, 10:22 AM | #2 |
Human being with feelings
Join Date: Dec 2012
Posts: 7,295
|
There is the Decorative Spectrum Analyzer preset included in the Video Processor. Doesn't look exactly like that, but it certainly could be hacked to do so. It is possible to make more sophisticated visualizers, too. I've messed around with some stuff, but some of things like people used to make for Winamp are kind of above my pay grade.
__________________
Lorenzo's Tractor is Everywhere --- Ash's Tube --- Join the Partnership for a Drum Free Amerika |
03-17-2021, 06:54 PM | #3 |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
You'd probably want a combination of a Video Processor preset to draw the "spectrum"
And a jsfx to to handle the audio (and do the analysis heavy lifting), passing an array of "levels" via gmem. If the OP's still interested I'll get something together.
__________________
it's meant to sound like that... |
03-17-2021, 07:41 PM | #4 |
Human being with feelings
Join Date: Dec 2012
Posts: 7,295
|
The preset that comes with Reaper gets the raw audio data via gmem from the JS video peeker plugin and then does the FFT stuff in the video processor itself. You certainly could do that FFT stuff in a JS plugin and then use gmem to send whatever you want. I'm not sure which way is better as far as CPU and stuff, but Justin wrote the existing code. Honestly, I think you can get pretty close to what the OP actually showed just by choosing proper settings in the existing preset, or at most it would take some pretty minor code tweaks.
__________________
Lorenzo's Tractor is Everywhere --- Ash's Tube --- Join the Partnership for a Drum Free Amerika |
03-18-2021, 07:06 AM | #5 | |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
Quote:
Indeed, and All Hail Justin, but this makes the visualisation component quite "heavy". (I don't even know how to import code into a preset, there must be a way? Is there an "//@import" ?) Also, developing and debugging Video Processor presets is a right pain, if they're more than trivial. If we wanted visualisation video plugins - one way of doing it would be to concoct a common jsfx backend, which exports higher level audio features (I guess via gmem), document that, and then anyone can write relatively lightweight visualisers (VP presets) using the features. e.g. I've got a jsfx which does a 12-band spectrum analysis - I can rejig it to put its measurements into gmem, and then the VP spectrum display preset is fairly simple. You could write a VP preset, using the same features in a different way, for an entirely different visualisation.
__________________
it's meant to sound like that... Last edited by jrk; 03-18-2021 at 08:32 AM. |
|
03-18-2021, 09:37 AM | #6 |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
OK, here's the spectrum "backend" jsfx.
It uses cookdsp, which you'd have to download & install yrself. http://ajaxsoundstudio.com/cookdspdoc/ Code:
desc: JRK Spectrum Backend options:gmem=JRK_spectrum import cookdsp.jsfx-inc slider1:0<-20,20,0.1>Adjustment (dB) slider2:2<1,4,1>Response slider3:1<1,10,1>Range @init adjustment = 1; response = 4; range = 5; mask = 0; // bandwidth 1.4 or so f1.bp(31.25, 1.4); f2.bp(62.5, 1.4); f3.bp(125, 1.4); f4.bp(250, 1.4); f5.bp(500, 1.4); f6.bp(1000, 1.4); f7.bp(2000, 1.4); f8.bp(4000, 1.4); f9.bp(8000, 1.4); f10.bp(12000, 2); f11.bp(16000, 2); f12.bp(20000, 2); level = memalloc(12); function exp_average(acc*, sc) ( alpha = 1 / (256*response); acc = (alpha * sc) + ( (1 - alpha) * acc) ; ); function set_op_level(b) ( z = (log10(level[b])) * -range; op_level = floor( z ) ; gmem[b] = op_level -1; ); function do_bands(mono) ( level[0] = exp_average(acc1,abs(f1.bp_do(mono))); level[1] = exp_average(acc2,abs(f2.bp_do(mono))); level[2] = exp_average(acc3,abs(f3.bp_do(mono))); level[3] = exp_average(acc4,abs(f4.bp_do(mono))); level[4] = exp_average(acc5,abs(f5.bp_do(mono))); level[5] = exp_average(acc6,abs(f6.bp_do(mono))); level[6] = exp_average(acc7,abs(f7.bp_do(mono))); level[7] = exp_average(acc8,abs(f8.bp_do(mono))); level[8] = exp_average(acc9,abs(f9.bp_do(mono))); level[9] = exp_average(acc10,abs(f10.bp_do(mono))); level[10] = exp_average(acc11,abs(f11.bp_do(mono))); level[11] = exp_average(acc12,abs(f12.bp_do(mono))); ); @slider adjustment = dbtoa(slider1); response = 2^slider2; range = slider3+4; @block i=0; while (i <12) ( set_op_level(i); i +=1; ); @sample do_bands((( spl0 + spl1 ) /2 ) * adjustment);
__________________
it's meant to sound like that... Last edited by jrk; 03-18-2021 at 01:21 PM. Reason: changed the response a bit... |
03-18-2021, 09:39 AM | #7 |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
... and here's the Video Processor preset
Code:
// eye candy spectrum //@gmem=JRK_spectrum col_w = 58; cell_h = 14; x = 4; start_y = y = 4; h = cell_h - y; w = col_w - (2*x); project_w = 12*col_w; project_h = (14*cell_h) + y; new_img = gfx_img_alloc(project_w,project_h,1); gfx_dest = new_img; i = 0; while (i<12) ( gfx_r = 1; gfx_g = 0; gfx_b = 0; gfx_a = 1; gfx_dest = new_img; gfx_fillrect(x, y, w, h); gfx_fillrect(x, y += cell_h , w, h); gfx_fillrect(x, y += cell_h , w, h); gfx_g = 1; gfx_fillrect(x, y += cell_h , w, h); gfx_fillrect(x, y += cell_h , w, h); gfx_fillrect(x, y += cell_h , w, h); gfx_fillrect(x, y += cell_h , w, h); gfx_r = 0; gfx_fillrect(x, y += cell_h , w, h); gfx_fillrect(x, y += cell_h , w, h); gfx_fillrect(x, y += cell_h , w, h); gfx_fillrect(x, y += cell_h , w, h); gfx_fillrect(x, y += cell_h , w, h); gfx_fillrect(x, y += cell_h , w, h); gfx_fillrect(x, y += cell_h , w, h); y = 0; // mask appropriately gfx_r = 0;gfx_g = 0;gfx_b = 0;gfx_a = 0.8; gfx_fillrect(x, y, w, (cell_h * gmem[i])); x += col_w; y = start_y; i+=1; ); gfx_a = 1; gfx_dest = -1; gfx_blit(new_img); gfx_img_free(new_img);
__________________
it's meant to sound like that... |
03-18-2021, 10:06 AM | #8 |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
...and here's a little clip of it in action (overlayed with the VP image overlay preset)
__________________
it's meant to sound like that... |
03-19-2021, 01:07 AM | #9 | |
Human being with feelings
Join Date: Dec 2012
Posts: 13,334
|
Quote:
Would you mind to post it there or allow me to do it? |
|
03-19-2021, 02:08 AM | #10 |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
Sure, I'll put a post together. It's really just a proof of concept - the basic idea is from the JS video peeker of course. But the approach does mean you can do the audio wrangling in a jsfx and just pass your important audio features to a (lightweight) VP preset.
__________________
it's meant to sound like that... |
03-19-2021, 02:20 AM | #11 | |
Human being with feelings
Join Date: Dec 2012
Posts: 13,334
|
Quote:
https://forum.cockos.com/showthread.php?t=237528 |
|
03-19-2021, 03:09 AM | #12 | |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
Quote:
__________________
it's meant to sound like that... |
|
03-19-2021, 08:46 AM | #13 |
Human being with feelings
Join Date: Dec 2012
Posts: 7,295
|
So one thing that the video peeker does is to send a sort of pointer value which I thought was because the video processor and audio engine are not actually synchronous so that it’s not otherwise possible to be sure that the video processor will be looking at the same sample that is actually playing at any given time. But then, it’s also dumping samples to gmem on buffer blocks, so maybe it’s just a way of navigating the audio block/video frame thing. IDK how much it matters in practice...???
I do have a preset out here that does zoom/Pan/opacity modulated by video peeker input. I also have a version of the oscilloscope which puts video from another track in the dots.
__________________
Lorenzo's Tractor is Everywhere --- Ash's Tube --- Join the Partnership for a Drum Free Amerika |
03-20-2021, 05:53 AM | #14 |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
I think video peeker also sends play_position, so with some clever business - comparing this to VP "project_time" - I guess it's possible to do something about synchronisation. And there's configurable "lookahead" which can probably help.
I'm not saying that I can do better than video peeker, it does what it does perfectly well. Rather, given the awkwardness of the VP preset "development environment", I'm arguing that it's easier (and more productive) to code the audio "heavy lifting" in jsfx. It also facilitates code reuse in a way that might support a wider "visualiser community". P.S. Of course, there's no reason why the communication has to be one-way - a VP could tell the jsfx where it's at, and the jsfx could delay or advance the audio features accordingly.
__________________
it's meant to sound like that... Last edited by jrk; 03-20-2021 at 06:49 AM. |
03-20-2021, 11:00 AM | #15 |
Human being with feelings
Join Date: Dec 2012
Posts: 7,295
|
No, I dig where you're coming from, and I don't mean to imply that you're doing it the wrong way. I frankly appreciate anybody working on developing these kinds of things because it is sorely needed and often requested.
I was kind of thinking, though, the video peeker is multichannel capable, and already built, and frankly you can pass any floating point numbers through it without really having to reinvent that part. Like, your plugin could just as easily spit out its values as "control voltages" on the audio channels, and video peeker will get it to gmem for you to do as you please. Then, too, other JS plugins could have access to that data. Not sure what you might do with it, but people might find uses like Parameter Modulation and stuff. Just a touch more modular and kind of saves some code in the analysis plugin itself.
__________________
Lorenzo's Tractor is Everywhere --- Ash's Tube --- Join the Partnership for a Drum Free Amerika |
03-20-2021, 01:13 PM | #16 | |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
Quote:
It's an accident that my example does it's levels (a) "upside down" and (b) quantised to 14 levels (it's cos I ripped it off from something else I'm doing). I'll rejig it so as to work in the way you suggest, as soon as I get time... It's more general.
__________________
it's meant to sound like that... |
|
03-20-2021, 01:21 PM | #17 |
Human being with feelings
Join Date: Dec 2012
Posts: 7,295
|
Course it might should come with a disclaimer to avoid sending those channels to your speakers, cause it probably won’t turn out very good.
__________________
Lorenzo's Tractor is Everywhere --- Ash's Tube --- Join the Partnership for a Drum Free Amerika |
03-22-2021, 10:00 AM | #18 |
Human being with feelings
Join Date: Feb 2021
Posts: 11
|
Thanks a lot. Great work. But how to install and run this script?
|
03-22-2021, 10:08 AM | #19 |
Human being with feelings
Join Date: Dec 2012
Posts: 13,334
|
It is not script. Reread the thread.
|
03-22-2021, 10:47 AM | #20 |
Human being with feelings
Join Date: Feb 2021
Posts: 11
|
|
03-22-2021, 11:09 AM | #21 |
Human being with feelings
Join Date: Dec 2012
Posts: 13,334
|
|
03-22-2021, 11:45 AM | #22 |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
Yep,
1. load in the Video processor preset (follow the link vitalker gave you for instructions). 2. Also ( on the same track is most logical ) you'll need the jsfx (a JS Plugin) "After downloading a JS plugin, add the files into the Effects folder of REAPER’s resource folder, then scan for new plugins. You can quickly find the resource folder from REAPER in the options menu, “Show REAPER’s resource path in explorer/finder…”" https://reaperblog.net/2015/06/quick...ll-js-plugins/ ( you might have to restart Reaper to get it to show up....) I've attached the file to this message for your convenience. 3. You'll also need the CookDSP JS library you can get this hereabouts http://ajaxsoundstudio.com/cookdspdoc/ Installation instructions are there also. 4. Route audio to this track (or just drop an audio item on it) 5. View video (of course)
__________________
it's meant to sound like that... |
03-22-2021, 12:04 PM | #23 |
Human being with feelings
Join Date: Feb 2021
Posts: 11
|
It does not work.
When i insert preset and click Ctrl+S, get error 1: syntax error: 'esc <!> : JRK Spectrum Backen' |
03-22-2021, 01:40 PM | #24 |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
The file I attached is a jsfx - NOT a Video Processor preset.
Download the file I attached up there... "SpectrumBackend.txt" Place it in your Reaper Effects folder. You ALSO need the video processor preset. That's up there at post 7
__________________
it's meant to sound like that... Last edited by jrk; 03-22-2021 at 01:47 PM. |
03-22-2021, 01:52 PM | #25 | |
Human being with feelings
Join Date: Feb 2021
Posts: 11
|
Quote:
Does not work. Works only Decorative spectrum Last edited by antoo; 03-22-2021 at 02:06 PM. |
|
03-22-2021, 02:33 PM | #26 |
Human being with feelings
Join Date: Feb 2021
Posts: 11
|
I have renamed SpectrumBackend.txt to SpectrumBackend.js. and it works. Thanks.
|
03-24-2021, 02:57 PM | #27 |
Human being with feelings
Join Date: Nov 2014
Location: Can Tho - Viet Nam
Posts: 305
|
Dear jrk,
How can we position Spectrum at will? Thanks
__________________
NK Recording Studio Email: lachinhan@gmail.com or admin@thuamninhkieu.com Website:nkpro.top and ntmusicpro.com |
03-25-2021, 09:46 AM | #28 |
Human being with feelings
Join Date: Feb 2021
Posts: 11
|
|
03-25-2021, 05:50 PM | #29 |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
Yes. That.
You'll often find that you have to "stack" Video Processor instances (with different presets) to get the effect you want. Of course, most Image Processors do just that - i.e. process an input image (or more than one). This one's a bit different, since it doesn't take any image input. Incidentally, the constants at the top Code:
col_w = 58; cell_h = 14; x = 4; start_y = y = 4;
__________________
it's meant to sound like that... Last edited by jrk; 03-25-2021 at 05:58 PM. |
03-25-2021, 06:25 PM | #30 | |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
Quote:
The more I look at this, the more I think it's arse up.
__________________
it's meant to sound like that... |
|
03-25-2021, 06:39 PM | #31 |
Human being with feelings
Join Date: Dec 2012
Posts: 7,295
|
Well with the Oscope, the sync is I think a little more important. Spectrum Analysis is window based anyway, so is never really sample-for-sample accurate. There’s not really any extra code for the preset in Oscope, though. Basically just use the position pointer as an index into the gmem buffer.
__________________
Lorenzo's Tractor is Everywhere --- Ash's Tube --- Join the Partnership for a Drum Free Amerika |
03-26-2021, 01:09 AM | #32 |
Human being with feelings
Join Date: Feb 2021
Posts: 11
|
Is it possible to show wave, during audio plays. File attached.
|
03-26-2021, 09:39 AM | #33 |
Human being with feelings
Join Date: Dec 2012
Posts: 7,295
|
Yes it’s possible. It’s not far off from what the Oscilloscope preset does, from a coding perspective. What I think I’m seeing in your picture, though, is more like if you were recording or maybe doing a real-time render and watching it draw the wave across the screen. It should be a fairly simple hack to what’s there, but I’m not going to get to that any time soon...
__________________
Lorenzo's Tractor is Everywhere --- Ash's Tube --- Join the Partnership for a Drum Free Amerika |
04-03-2021, 07:52 AM | #34 |
Human being with feelings
Join Date: Feb 2021
Posts: 11
|
Where in settings is possible to change video delay, because Spectr goes for a few seconds with delay?
|
04-06-2021, 02:12 AM | #35 |
Human being with feelings
Join Date: Aug 2015
Posts: 2,969
|
The easiest way is to put the backend jsfx on a track with a copy of the audio. Then manually drag that audio leftwards (earlier) so that the video lines up with the audio.
If you do this, you wouldn't route the "early" audio to the master. Alternatively - and again, you'd want the backend on another track, probably routing the audio to it... Here's a version (of the backend jsfx) that adds lookahead. Code:
desc: JRK Spectrum Backend options:gmem=JRK_spectrum import cookdsp.jsfx-inc slider1:0<-20,20,0.1>Adjustment (dB) slider2:2<1,4,1>Response slider3:1<1,10,1>Range slider4:lookahead_seconds=0.5<0,1.8,0.1>lookahead (seconds) @init adjustment = 1; response = 4; range = 5; mask = 0; // bandwidth 1.4 or so f1.bp(31.25, 1.4); f2.bp(62.5, 1.4); f3.bp(125, 1.4); f4.bp(250, 1.4); f5.bp(500, 1.4); f6.bp(1000, 1.4); f7.bp(2000, 1.4); f8.bp(4000, 1.4); f9.bp(8000, 1.4); f10.bp(12000, 2); f11.bp(16000, 2); f12.bp(20000, 2); pdc_bot_ch=0; pdc_top_ch=2; level = memalloc(12); function exp_average(acc*, sc) ( alpha = 1 / (256*response); acc = (alpha * sc) + ( (1 - alpha) * acc) ; // ret = acc + ((sc>acc)?(sc/10):0); // biases towards attack... ); function set_op_level(b) ( z = (log10(level[b])) * -range; op_level = floor( z ) ; gmem[b] = op_level -1; ); function do_bands(mono) ( level[0] = exp_average(acc1,abs(f1.bp_do(mono))); level[1] = exp_average(acc2,abs(f2.bp_do(mono))); level[2] = exp_average(acc3,abs(f3.bp_do(mono))); level[3] = exp_average(acc4,abs(f4.bp_do(mono))); level[4] = exp_average(acc5,abs(f5.bp_do(mono))); level[5] = exp_average(acc6,abs(f6.bp_do(mono))); level[6] = exp_average(acc7,abs(f7.bp_do(mono))); level[7] = exp_average(acc8,abs(f8.bp_do(mono))); level[8] = exp_average(acc9,abs(f9.bp_do(mono))); level[9] = exp_average(acc10,abs(f10.bp_do(mono))); level[10] = exp_average(acc11,abs(f11.bp_do(mono))); level[11] = exp_average(acc12,abs(f12.bp_do(mono))); ); @slider adjustment = dbtoa(slider1); response = 2^slider2; range = slider3+4; pdc_delay=(lookahead_seconds*srate)|0; @block i=0; while (i <12) ( set_op_level(i); i +=1; ); @sample do_bands((( spl0 + spl1 ) /2 ) * adjustment);
__________________
it's meant to sound like that... |
Thread Tools | |
Display Modes | |
|
|