Old 01-09-2021, 12:01 PM   #1
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,272
Default 64 channel video peeker and adapted oscilloscope

An issue I've run into, and that came up on the FB group yesterday, is that the basic JS video sample peeker is basically just stereo and is global so that you can't have two different oscilloscopes (or whatever) follow two different audio tracks without a whole lot of messing around.


So, this is actually abusive and probably voids your Reaper warranty. It will not work with sample rates much bigger than 48K, but it actually does work, and expands it from the 2 channel stereo to a full 64 channels. The idea being that each audio track could send to a different pair of channels and each video processor preset could choose which pair to look at.



64 channel video sample peeker:
Code:
desc:64 channel video sample peeker
// copies samples to shared global memory for use by some video processor presets

slider1:lookahead_seconds=1<0,1.8,0.1>lookahead (seconds, 1.0 is normal)
options:gmem=ashcat_jsfx_to_video



@init
send_nch=64;
pos=bufstart=16;
bufend=bufstart+((2.0 /*seconds*/ *srate)|0)*send_nch;

@slider
pdc_delay=(lookahead_seconds*srate)|0;

@block
gmem[0]=play_position;
gmem[1]=pos;
gmem[2]=srate;
gmem[3]=bufstart;
gmem[4]=bufend;
gmem[5]=send_nch;
  
@sample
i = 0;
while (i < send_nch)
  (gmem[pos+i]  =spl(i);
   i += 1;
  );
//gmem[pos+1]=spl1;
(pos+=send_nch) >= bufend ? pos=bufstart;
adapted oscilloscope:
Code:
//Decorative Oscilloscope with Blitter (from 64 channel peeker)
//@gmem=ashcat_jsfx_to_video
//@param 1:mode "mode" 0 0 2 0 1
//@param 2:dotcount "point count" 1200 10 5000 400 10
//@param 3:dotsize "point size" 4 2 40 20 1
//@param 4:gain_db "gain (dB)" -6 -80 12 -12 1
//@param 5:zoom_amt "blitter zoom" .27 -.5 1 .1 0.01
//@param 6:fadespeed "blitter persist" .8 0 1 .1 0.01
//@param 7:filter "blitter filter" 1 0 1 0 1
//@param 8:fg_r "foreground R" 1 0 1 .5 .02
//@param 9:fg_g "foreground G" 1 0 1 .5 .02
//@param 10:fg_b "foreground B" 1 0 1 .5 .02
//@param 11:bg_r "background R" 0 0 1 .5 .02
//@param 12:bg_g "background G" 0 0 1 .5 .02
//@param 13:bg_b "background B" 0 0 1 .5 .02
//@param 14:cx "center X" .5 0 1 .5 .01
//@param 15:cy "center Y" .5 0 1 .5 .01
//@param 16:in_pair "channel pair (left=)" 1 1 63 32 2

last_frame && fadespeed > 0 ? (
  xo = project_w*zoom_amt*.25;
  yo = project_h*zoom_amt*.25;
  gfx_mode=filter>0?0x100:0;
  xo < 0 ? gfx_blit(last_frame,0);
  gfx_blit(last_frame,0,0,0,project_w,project_h,xo,yo,project_w-xo*2,project_h-yo*2);
);
gfx_set(bg_r,bg_g,bg_b,last_frame ? (1-fadespeed) : 1);
gfx_a>.001 ? gfx_fillrect(0,0,project_w,project_h);

bufplaypos = gmem[0];
bufwritecursor = gmem[1];
bufsrate = gmem[2];
bufstart = gmem[3];
bufend = gmem[4];
nch = gmem[5];
gain = 10^(gain_db*(1/20));
pair = in_pair - 1;

dt=max(bufplaypos - project_time,0);
dt*bufsrate < dotcount ? underrun_cnt+=1;
rdpos = bufwritecursor - ((dt*bufsrate - dotcount)|0)*nch;
rdpos < bufstart ? rdpos += bufend-bufstart;

gfx_set(fg_r,fg_g,fg_b,1);

function getpt()
(
  l = gmem[rdpos+pair]; r = gmem[rdpos+pair+1];
  (rdpos += nch)>=bufend ? rdpos=bufstart;
);
i=0;

mode==2 ? (
  loop(dotcount,
    getpt();
    ang = atan2(l,r); dist = sqrt(sqr(l)+sqr(r));
    xp = cx*project_w + ((cos(ang)*dist*gain)*project_h-dotsize)*.5;
    yp = ((cy*2+sin(ang)*dist*gain)*project_h-dotsize)*.5;
    gfx_fillrect(xp,yp, dotsize,dotsize);
  );
) : mode == 1 ? (
  loop(dotcount,
    getpt();
    yp = project_h * (cy - .5 + i / dotcount);
    xp = project_w * (cx + (l+r)*.25*gain);
    gfx_fillrect(xp-dotsize*.5,yp-dotsize*.5, dotsize,dotsize);
    i+=1;
  );

) : (
  loop(dotcount,
    getpt();
    xp = project_w * (cx - 0.5 + i / dotcount);
    yp = project_h * (cy + (l+r)*.25*gain);
    gfx_fillrect(xp-dotsize*.5,yp-dotsize*.5, dotsize,dotsize);
    i+=1;
  );
);

gfx_img_free(last_frame);
last_frame=gfx_img_hold(-1);
I had trouble getting the spectrogram analyzer preset to work with this, but didn't really try that hard. If anybody figures that out, let us know!

Last edited by ashcat_lt; 03-27-2022 at 12:56 PM. Reason: for some reason it wasn't syncing correctly. seems to work now.
ashcat_lt is online now   Reply With Quote
Old 01-09-2021, 12:53 PM   #2
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

Probably would be good to ensure that the buffer length is a multiple of the channel count (in case 2 seconds worth of samples isn't a multiple of 64):
Code:
bufend=bufstart+((2.0 /*seconds*/ *srate)|0)*send_nch;

There are some bugs in the spectrogram, I'm fixing these for the next build, here's a fixed one that works with (the hopefully fixed) 64-channel peeker:
Code:
// Decorative Spectrum Analyzer (requires JSFX video sample peeker)
//@gmem=ashcat_jsfx_to_video
//@param 1:scroll_duration 'scroll duration' 8 1 30 15 0.1
//@param 2:gram_size_ratio 'spectrogram size' 0.85 0 1 0.5 .01
//@param 3:channel_mode 'chan (2=stereo, 3=mono mix)' 2 0 3 1 1
//@param 4:fft_size_req 'FFT size (bits)' 10 5 15 10 1

//@param 6:use_smooth 'Improved sampling' 0 0 1 0 1
//@param 7:use_log 'Logarithmic' 0 0 8 0 1

//@param 9:analyzer_hue 'Analyzer hue' -20 -360 360 0 1
//@param 10:analyzer_hue_range 'Analyzer hue range' 80 -360 360 0 1

//@param 11:gram_hue "'gram hue" 0 -360 360 0 1
//@param 12:gram_hue_range "'gram hue range" 240 -720 720 0 1

//@param 16:in_pair "channel pair (left=)" 1 1 63 32 2

colorspace='RGBA';

fft_size=min(max(1<<fft_size_req,32),32768);
(meter_size = (project_h*(1-gram_size_ratio))|0) < 20 ? meter_size=0;
meter_size > project_h-20 ? meter_size=project_h;
gram_size = project_h-meter_size - (meter_size>0?floor(project_h/32));

bufplaypos = gmem[0];
bufwritecursor = gmem[1];
bufsrate = gmem[2];
bufstart = gmem[3];
bufend = gmem[4];
nch = gmem[5];

scroll_size = max((project_w/(scroll_duration*framerate))|0,1);
use_amt = ((bufsrate / framerate)|0) + fft_size;

dt=max(bufplaypos - project_time,0);
dt*bufsrate < use_amt ? underrun_cnt+=1;
rdpos = bufwritecursor - ((dt*bufsrate - use_amt)|0)*nch;
rdpos < bufstart ? rdpos += bufend-bufstart;

left_buf = 0;
right_buf = left_buf + fft_size;
window_buf = right_buf + fft_size;

window_size != fft_size ? (
  window_size=0;
  loop(fft_size,
    wp = window_size*(2*$pi)/fft_size;
    window_buf[window_size] = (0.35875 - 0.48829 * cos(wp) +
                               0.14128 * cos(2*wp) - 0.01168 * cos(3*wp));
    window_size += 1;
  );
);

function h6s2i(h) global() ((h -= floor(h*1/6)*6)<3 ? h<1 ? 1-h : 0 : h<4 ? h-3 : 1);
function set_hsv(h,s,v) global() (
  h*=1/60; s *= v;
  gfx_set(v-h6s2i(h+2)*s,v-h6s2i(h)*s,v-h6s2i(h-2)*s);
);

function analyze(gmemrdpos, buf, fft_size) local(i w n offs v)
  global(gmem window_buf nch bufend bufstart channel_mode in_pair) (
  i = 0;
  channel_mode==1? gmemrdpos += 1;
  loop(fft_size,
    gmemrdpos >= bufend ? gmemrdpos+=bufstart-bufend;
    v = gmem[gmemrdpos+in_pair-1];
    channel_mode==3 ? v = (v+gmem[gmemrdpos+1+in_pair-1])*.5;
    buf[i] = v * window_buf[i];
    gmemrdpos += nch;
    i+=1;
  );
  fft_real(buf,fft_size);
  fft_permute(buf,fft_size/2);

  offs = -log(fft_size*.25)*2;
  w=0;
  i=2;
  v = sqr(buf[1]);
  n = v < 10^-20 ? -200 : (log(v)+offs)*(10.0/log(10)); // save nyquist
  loop(fft_size*.5 - 1,
    v=sqr(buf[i])+sqr(buf[i+1]);
    buf[w]=v < 10^-20 ? -200 : (log(v)+offs) * (10.0/log(10));
    w+=1;
    i+=2;
  );
  buf[w]=n;
);

log_scale = use_log ? 1/(exp(1+use_log)-$e);
function logscale(f) global(use_log log_scale) (
  use_log ? (exp(1+f*use_log) - $e) * log_scale: f;
);

function sample(buf, f, df, bufsz) global(use_smooth) local(next rdidx frac) (
  frac = logscale(f)*bufsz;
  rdidx = frac|0;
  use_smooth ? (
    frac -= rdidx;
    next = (logscale(f+df)*bufsz)|0;
    next == rdidx && rdidx<bufsz-1 ? buf[rdidx] * (1-frac) + buf[rdidx+1]*frac :
    next <= rdidx+1 ? buf[frac<.5||next>=bufsz?rdidx:next] : (
      frac=buf[rdidx];
      loop(next-rdidx-1, frac = max(frac,buf[rdidx+=1]); );
      frac;
    );
  ) : buf[rdidx];
);

function draw_analyzer(buf, bufsz, xpos, ypos, w, h)
  global(gfx_dest analyzer_hue analyzer_hue_range)
  local(meter_image meter_image_size i di yh hue_range hue_start) (
  meter_image_size != h || hue_start != analyzer_hue || hue_range != analyzer_hue_range ? (
    meter_image_size = h;
    hue_start = analyzer_hue;
    hue_range = analyzer_hue_range;
    gfx_img_free(meter_image);
    gfx_dest = meter_image = gfx_img_alloc(2,(h+1)&0xffffe);
    i=0;
    loop(h,
      set_hsv(i*analyzer_hue_range/h+analyzer_hue,.7,.9);
      gfx_fillrect(0,i,2,2);
      i+=1;
    );
    gfx_dest = -1;
  );

  i=0; di = 1/w;
  loop(w,
    yh = 0|(min(max(0,((sample(buf,i,di,bufsz)*(1/70))+1)),1)*h);
    yh>0?gfx_blit(meter_image, 0, xpos, ypos-yh, 1, yh, 0, h-yh, 1, yh);
    i+=di;
    xpos+=1;
  );
);

function draw_gram(buf, bufsz, xpos, ypos, w, h)
  local(i di v) global(gram_hue gram_hue_range) (
  i=0; di = 1/h;
  loop(h,
    v = 2+sample(buf,i,di,bufsz)*1/64;
    set_hsv(gram_hue + v*gram_hue_range,.7,min(max(v*2-1,0),1)*(.5+v*.25));
    gfx_fillrect(xpos,ypos-=1,4,1);
    i+=di;
  );
);

last_frame && last_frame_w == project_w && last_frame_h == project_h ? (
  gfx_blit(last_frame,0, 0,0,project_w-scroll_size,gram_size,
                         scroll_size,0, project_w-scroll_size,gram_size);
  gfx_fillrect(project_w-scroll_size,0,scroll_size,gram_size);
  gfx_fillrect(0,gram_size,project_w,project_h-gram_size);
) : gfx_fillrect(0,0,project_w,project_h);

sp = 0;
drdpos = ((use_amt-fft_size)/scroll_size)*nch;
loop(gram_size>0 ? scroll_size : 1,
  sp==0 || (rdpos&0xffffffe) != lrdpos ? (
    lrdpos = (rdpos&0xffffffe);
    analyze(lrdpos,left_buf,fft_size);
    channel_mode==2?analyze(lrdpos+1,right_buf, fft_size);
  );
  rdpos += drdpos;
  gram_size>0 ? (
    sz = channel_mode==2 ? gram_size*.5 : gram_size;
    draw_gram(left_buf,fft_size*.5, project_w-scroll_size+sp, sz, 1, sz);
    channel_mode==2 ?
      draw_gram(right_buf,fft_size*.5, project_w-scroll_size+sp, gram_size, 1, gram_size*.5);
  );
  sp+=1;
);

meter_size>0 ? (
  // draw analyzer from last analysis
  draw_analyzer(left_buf,fft_size*.5, 0, project_h, channel_mode==2 ? project_w*.5 - 4:project_w, meter_size);
  channel_mode==2?
    draw_analyzer(right_buf,fft_size*.5, project_w*.5 + 4, project_h, project_w*.5-4, meter_size);
);

gfx_img_free(last_frame);
last_frame=gfx_img_hold(-1);
last_frame_w = project_w;
last_frame_h = project_h;

Last edited by Justin; 01-09-2021 at 01:09 PM.
Justin is offline   Reply With Quote
Old 01-09-2021, 02:46 PM   #3
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,272
Default

Thanks Justin! That spectrogram sure does seem to work in quick tests.


I guess I don't see how this

Quote:
Originally Posted by Justin View Post
Code:
 bufend=bufstart+((2.0 /*seconds*/ *srate)|0)*send_nch;
is functionally different from what was already in the original sample peeker
Code:
bufend=bufstart+((2.0 /*seconds*/ *srate*send_nch)|0)
but I guess I'm a little iffy on the bitwise operators sometimes...
ashcat_lt is online now   Reply With Quote
Old 01-10-2021, 09:12 AM   #4
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

Quote:
Originally Posted by ashcat_lt View Post
Thanks Justin! That spectrogram sure does seem to work in quick tests.


I guess I don't see how this


is functionally different from what was already in the original sample peeker
Code:
bufend=bufstart+((2.0 /*seconds*/ *srate*send_nch)|0)
but I guess I'm a little iffy on the bitwise operators sometimes...
It won't differ in practice, but if you change the 2.0 to a non-whole number or run at a strange fractional samplerate it might.

Let's dissect this:
Code:
(2.0 *srate*send_nch)|0
Under a real scenario, this does 2.0 * 44100 * 64, then rounds that result down to an integer. 2*44100 is an integer, so the result will be a multiple of 64 (send_nch), which is what we want.

Suppose you run the samplerate at 44100.1, and change the factor of 2.0 to 1.9. Then you'd have 44100.1 * 1.9 * 64, which is 5362572.16. Doing |0 would round it to 5,362,572, which isn't a multiple of 64.

If you do ((1.9*44100.1)|0)*64, the initial multiplication would be 83,790.19, rounded to 83,790, then multiplied by 64, or 5,362,560, which is a multiple of 64.

Anyway probably somewhat moot but good to have that logic in a more correct way in case some odd scenario comes about
Justin is offline   Reply With Quote
Old 01-10-2021, 10:46 AM   #5
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,272
Default

Ah yeah that makes sense. Thanks.
ashcat_lt is online now   Reply With Quote
Old 02-10-2023, 01:09 PM   #6
BrianKeesbury
Human being with feelings
 
Join Date: Feb 2009
Location: Ohio
Posts: 6
Default

Hi,

I'm trying to work on setting this up but have been unable to get it to work. Do you have a quick guide how this is set up compared to the original?
BrianKeesbury is offline   Reply With Quote
Old 02-10-2023, 01:39 PM   #7
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,272
Default

I'd be happy to help, but from my perspective, it seems so basic and simple that I'm not sure where to start. Put the JS plugin somewhere, route audio to various channels, put the vp preset somewhere, tell it which channel to look at. But I guess that assumes a reasonable level of familiarity with any of these things, and I'm not sure where you're at on any of it. If you let me know what part you're stuck on, I'm sure we can sort it out.
ashcat_lt is online now   Reply With Quote
Old 03-09-2023, 01:29 PM   #8
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,272
Default

How big does this buffer really need to be?!?

As it is, it passes 2 full seconds worth of samples per channel. Or it wants to anyway. But like at 48K sample rate, and 64 channels, that’s over 6 million slots! Any higher sample rate is going to put us over the “around 8 million” cap on our private gmem buffer. So….

I think that part of the point of buffering more than one sample per channel is to allow some slop in the timing between the video and audio threads, but I find it difficult to believe that alone needs 2 full seconds.

So at least part of the point in this implementation is literally to give the oscope more than one dot to draw, and the spectrum dude enough samples to do FFT stuff with.

But in many instances where we want to use audio to manipulate parameters in a preset, (using audio as CV) we don’t really need all that. We still probably (?) want enough history for the first part - slop between threads - but…

Well, how bad do we expect that to get? I suppose it depends on just about everything. How much processing we’re actually doing in each thread and so on. Is there any way to even estimate a safe minimum length for this?

Truth bet told, I have other reasons for asking, but it does concern me that this will break at higher sample rates.
ashcat_lt is online now   Reply With Quote
Old 03-29-2023, 04:23 AM   #9
Medialink
Human being with feelings
 
Join Date: Jan 2017
Posts: 4
Default

Quote:
Originally Posted by ashcat_lt View Post
64 channel video sample peeker:
Code:
desc:64 channel video sample peeker
// copies samples to shared global memory for use by some video processor presets

slider1:lookahead_seconds=1<0,1.8,0.1>lookahead (seconds, 1.0 is normal)
options:gmem=ashcat_jsfx_to_video



@init
send_nch=64;
pos=bufstart=16;
bufend=bufstart+((2.0 /*seconds*/ *srate)|0)*send_nch;

@slider
pdc_delay=(lookahead_seconds*srate)|0;

@block
gmem[0]=play_position;
gmem[1]=pos;
gmem[2]=srate;
gmem[3]=bufstart;
gmem[4]=bufend;
gmem[5]=send_nch;
  
@sample
i = 0;
while (i < send_nch)
  (gmem[pos+i]  =spl(i);
   i += 1;
  );
//gmem[pos+1]=spl1;
(pos+=send_nch) >= bufend ? pos=bufstart;
When I try to save this script it's giving the following error msg -

'Line 1: <name> expected near '64'
Medialink is offline   Reply With Quote
Old 03-29-2023, 09:08 AM   #10
ashcat_lt
Human being with feelings
 
Join Date: Dec 2012
Posts: 7,272
Default

Seems like you missed “desc:” at the beginning??? Anyway, this .zip includes the JS plugin and a sample project showing one possible way of using it.

https://stash.reaper.fm/46638/4%20ch...pe%20video.zip

Edit - actually it seems to save with or with desc:, so idk what’s happening on your end.

Last edited by ashcat_lt; 03-29-2023 at 11:52 AM.
ashcat_lt is online now   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 01:58 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.