Old 05-12-2022, 03:38 PM   #1
juan_r
Human being with feelings
 
juan_r's Avatar
 
Join Date: Oct 2019
Posts: 1,075
Default New MIDI JSFX - Floating (smart) split point

((EDIT - I've copy-pasted a preliminary version further down in this thread.
It is here: https://forum.cockos.com/showpost.ph...22&postcount=8
Eventually this JSFX should end up on ReaPack.))


I'm considering giving a shot to a potentially very useful JSFX for MIDI processing that implements floating (or smart) split, where the keyboard is split between left hand and right hand - say, bass and piano respectively, but the split point tracks the performance in some way.

This would make it possible to go low with the piano, or high with the bass, while still keeping the two parts on separate channels.

Of course it's going to be a hit-and-miss thing - not a surefire solution, but even so it could be quite useful.

I'm thinking of several bits of data that could be relevant:

- minimum separation between hands (LH highest/RH lowest notes)
- minimum (maximum?) number of voices on each side. E.g., bass (LH) has one note at most
- ??

I'd like to keep it simple at first, see if it works, and then possibly improve from a basic working script.

Any idea for an algorithm that makes sense?

Last edited by juan_r; 05-18-2022 at 04:55 PM. Reason: Changed title again, linked to preliminary version
juan_r is offline   Reply With Quote
Old 05-16-2022, 02:42 PM   #2
juan_r
Human being with feelings
 
juan_r's Avatar
 
Join Date: Oct 2019
Posts: 1,075
Default

Replying rather than editing the OP.

I've got all the logic in place, and the plugin works except it's not moving the split point around yet. However, it is tracking the left and right hand ranges independently, and the number of voices in each hand. So what could be done with this information to get a smart floating split point?

Simple rules would be welcome. I'm looking for suggestions. Brainstorm anyone?
juan_r is offline   Reply With Quote
Old 05-16-2022, 03:41 PM   #3
jrk
Human being with feelings
 
Join Date: Aug 2015
Posts: 2,969
Default

I suppose you've thought of this, but what you're looking at here is a bimodal distribution sort of thing. If you consider it as a thresholding problem, something like Otsu's method might be valuable. Or a clustering problem - k-means (k = 2) Or something like that.

But in real-time, live? Hmmm...
__________________
it's meant to sound like that...
jrk is offline   Reply With Quote
Old 05-16-2022, 04:09 PM   #4
juan_r
Human being with feelings
 
juan_r's Avatar
 
Join Date: Oct 2019
Posts: 1,075
Default

Thanks for throwing an idea into the box. I see where you're going, but I don't want to go there! I'm looking for some deterministic rules that are not too hard to code in EEL2.

Real-time, and somewhat fallible, will likely be good enough.
By "fallible" I mean that the system can be made to fail, but if one plays it "right" it will behave predictably.

Something along the lines of:

Extend the last hand that played, as long as
1. The new note is closer to the last hand than to the other hand
2. The resulting split point movement doesn't exceed a user-set threshold
3. The resulting spread of the playing hand doesn't exceed a user-set hand size

In Point 1, by "closer" I mean closer to the closest note (which is going to be one of the hand extremes), not closer to the "center of the distribution", however it may be defined. I think what matters is what the hands have been doing lately, not the whole history.

1., 2., 3. (or similar 'clever' rules I'm trying to conjure) could be alternatives, or OR-ed/AND-ed in some combination.

This is the kind of reasoning I'm trying to focus on - and asking for help about.
juan_r is offline   Reply With Quote
Old 05-16-2022, 11:54 PM   #5
jrk
Human being with feelings
 
Join Date: Aug 2015
Posts: 2,969
Default

Well, it is an adaptive threshold kind of problem.
Simply:
Code:
// initialise 
threshold = 63;					// the split point


// in a loop for each new note
(
	LHcentre = threshold / 2;
	RHcentre = ((127 - threshold)/2) + threshold;

	newnote < threshold ? // Left hand
	(
		newnote < LHcentre ?
		(
			// lower threshold
		);
		newnote > LHcentre ?
		(
			// raise threshold
		);
		
	): // Right hand
	(
		newnote < RHcentre ?
		(
			// lower threshold
		);
		newnote > RHcentre ?
		(
			// raise threshold
		);
	);
);
Your other heuristics might go in your raise / lower threshold function
__________________
it's meant to sound like that...

Last edited by jrk; 05-17-2022 at 12:01 AM.
jrk is offline   Reply With Quote
Old 05-17-2022, 04:07 AM   #6
juan_r
Human being with feelings
 
juan_r's Avatar
 
Join Date: Oct 2019
Posts: 1,075
Default

The code you propose moves the split point more when one hand goes in "its" direction more - that is, when the left plays deep bass, or the right plays very high. In such case there's not that much need to move the split point. My efforts are on the other side: that of notes that are near the present split point.

My goal is to let one "line" go on unobstructed by capturing continuity. Yes, meaningless qualitative words... that's why I'm proposing this brainstorm. Your contribution is welcome and I thank you for stirring the pot!

EDIT
A preliminary version is working, not perfect but fine. A few corner cases will need to be ironed out, but already it moves the split point over more than an octave in a usable, playable way. Not sure which kind of logic I could add to improve the experience. I'm playing with it, letting it sit and brew for a while.

Last edited by juan_r; 05-17-2022 at 10:10 AM. Reason: Reporting progress
juan_r is offline   Reply With Quote
Old 05-17-2022, 10:31 AM   #7
jrk
Human being with feelings
 
Join Date: Aug 2015
Posts: 2,969
Default

Quote:
Originally Posted by juan_r View Post
The code you propose moves the split point more when one hand goes in "its" direction more - that is, when the left plays deep bass, or the right plays very high. In such case there's not that much need to move the split point. My efforts are on the other side: that of notes that are near the present split point.
No, it needn't move the threshold more if (say) a note is an octave below the LH centre than when a semitone below. Also, it needn't move the threshold (split point) much at a time. Equally, the LH branch needn't move as much as the RH.

But it sounds like you've figured out a way of doing what you want. Thank you for the stimulating question.
__________________
it's meant to sound like that...
jrk is offline   Reply With Quote
Old 05-18-2022, 04:52 PM   #8
juan_r
Human being with feelings
 
juan_r's Avatar
 
Join Date: Oct 2019
Posts: 1,075
Default

So here's the first version that feels usable to me.

To use: put this on a track and send LH/RH MIDI channels to two other tracks, each with a single VSTi for left or right hand respectively.

Hint: Try it with high values for the Max UP and Max DOWN movement parameters: 24 semitones or almost. You can type any value, even beyond 24. Such large values do work well.

Comments and suggestions welcome!

Code:
desc:MIDI Floating Split
author: Juan_R
version: 0.02a

changelog:
  0.01a  initial release
  0.02a  it works but still needs some knowing it.
about:
# MIDI Floating Split

Move the splitpoint along, tracking both hands.

This is alpha.
No hurry to actually see it committed before it goes out of alpha.

Hoping to improve it.
More sophisticated heuristics anyone?
The plug's collecting some data that isn't used yet.

I'm submitting this as the snapshot of a working version:
Something to get back to if experiments go astray.
Todo - Manage one hand left alone corner case more gracefully.

Presets would be helpful.
Switching presets = change to a slightly different FA.
Some automated switching? Expose switching parameters?


// No audio input/output: this is a MIDI-only FX. Let Reaper know that.
in_pin:none
out_pin:none

slider1:2<1,16,1>Left  Hand channel
slider2:LHS=17<1,24,1>-Left hand max span (semitones)
slider3:MAXUP=5<0,24,1>Split point max UP movement
slider4:LHT=0<-88,88,1>Left  Hand transpose
slider5:ISP=48<0,127,1{0: C-1,1: C#-1,2: D-1,3: D#-1,4: E-1,5: F-1,6: F#-1,7: G-1,8: G#-1,9: A-1,10: A#-1,11: B-1,12: C0,13: C#0,14: D0,15: D#0,16: E0,17: F0,18: F#0,19: G0,20: G#0,21: A0,22: A#0,23: B0,24: C1,25: C#1,26: D1,27: D#1,28: E1,29: F1,30: F#1,31: G1,32: G#1,33: A1,34: A#1,35: B1,36: C2,37: C#2,38: D2,39: D#2,40: E2,41: F2,42: F#2,43: G2,44: G#2,45: A2,46: A#2,47: B2,48: C3,49: C#3,50: D3,51: D#3,52: E3,53: F3,54: F#3,55: G3,56: G#3,57: A3,58: A#3,59: B3,60: C4,61: C#4,62: D4,63: D#4,64: E4,65: F4,66: F#4,67: G4,68: G#4,69: A4,70: A#4,71: B4,72: C5,73: C#5,74: D5,75: D#5,76: E5,77: F5,78: F#5,79: G5,80: G#5,81: A5,82: A#5,83: B5,84: C6,85: C#6,86: D6,87: D#6,88: E6,89: F6,90: F#6,91: G6,92: G#6,93: A6,94: A#6,95: B6,96: C7,97: C#7,98: D7,99: D#7,100: E7,101: F7,102: F#7,103: G7,104: G#7,105: A7,106: A#7,107: B7,108: C8,109: C#8,110: D8,111: D#8,112: E8,113: F8,114: F#8,115: G8,116: G#8,117: A8,118: A#8,119: B8,120: C9,121: C#9,122: D9,123: D#9,124: E9,125: F9,126: F#9,127: G9}>Initial split point
slider6:1<1,16,1>Right Hand channel
slider7:RHS=17<1,24,1>-Right hand max span (semitones)
slider8:MAXDOWN=5<0,24,1>Split point max DOWN movement
slider9:RHT=0<-88,88,1>Right Hand transpose

@init

// Playing channels for all pressed keys. LHC, LRC (0-15) or -1 if key not pressed
KeyChannels = 0;                                    // array location [0, 127] => end at 127
KeyChannelsSize = 128;                              // array size for mem alloc

// Output notes played for all playing keys. -1 = none (not playing)
KeyNotes = KeyChannels + KeyChannelsSize;           // array location
KeyNotesSize = 128;                                 // array size for mem alloc

LastMem = KeyNotes + KeyNotesSize;                  // Last mem location

i = 0;                                              // init all channels and notes to -1
while (i < KeyChannelsSize) (
        KeyChannels[i] = KeyNotes[i] = -1;
        i += 1;
);

LHL = RHL = 128;                                    // LH, RH lowest key: init out of range
LHH = RHH = -1;                                     // LH, RH highest key: init out of range
LHV = 0;                                            // Left Hand Voices - how many playing?
RHV = 0;                                            // Right Hand Voices - how many playing?
lasthandon = -1;                                    // last hand that played note on. 0 = left, 1 = right
lasthandoff = -1;                                   // last hand that played note off. 0 = left, 1 = right
lastLH = -1;                                        // last note played by LH
lastRH = -1;                                        // last note played by RH

function is_note(status) (
    status == 0x90 || status == 0x80;
);

function is_not_note(status) (
    status != 0x90 && status != 0x80;
);

function is_noteon(status, vel) (
    status == 0x90 && vel > 0;
);

function is_noteoff(status, vel) (
    status == 0x80 || (status == 0x90 && vel == 0);
);

// re-calculate hand bounds
// function recalc_bounds() - split functionally into symmetrical LH, RH parts
// nnote: the note number triggering recalc. on_off==1 if note on, on_off==0 if note off.

function recalc_bounds_LH(nnote, on_off)
local (i done) (
    done = 0;
    (nnote > LHL && nnote < LHH) ? done = 1;                        // no changes, we're done - be it note on or note off
    (!done && on_off == 1) ? (                                      // note ON
        (nnote >= LHH) ? (LHH = nnote; done = 1);
        (nnote <= LHL) ? (LHL = nnote; done = 1);
    ) :
    (                                                               // else, it's a note OFF
        (LHV == 0) ? (LHH = -1; LHL = 128; done = 1);               // no voices playing
        (!done && nnote == LHH) ? (                                 // note OFF top note, not last voice, scan down for new high LHH
            i = nnote;
            while (!done && i >= 0)
            (
                (KeyChannels[i] == LHC) ? (LHH = i; done = 1;);
                i -= 1;
            );
        )
        : (nnote == LHL) ? (                                        // note OFF bottom note, not last voice, scan up for new low LHL
            i = nnote;
            while (!done && i < 128)
            (
                (KeyChannels[i] == LHC) ? (LHL = i; done = 1;);
                i += 1;
            );
        ); // end (nnote == LHL) ?
    ); // end "else it's a note OFF"
);

function recalc_bounds_RH(nnote, on_off)
local (i done) (
    done = 0;
    (nnote > RHL && nnote < RHH) ? done = 1;                        // no changes, we're done - be it note on or note off
    (!done && on_off == 1) ? (                                      // note ON
        (nnote >= RHH) ? (RHH = nnote; done = 1);
        (nnote <= RHL) ? (RHL = nnote; done = 1);
    ) :
    (                                                               // else, it's a note OFF
        (RHV == 0) ? (RHH = -1; RHL = 128; done = 1);               // no voices playing
        (!done && nnote == RHH) ? (                                 // note OFF top note, not last voice, scan down for new high RHH
            i = nnote;
            while (!done && i >= 0)
            (
                (KeyChannels[i] == RHC) ? (RHH = i; done = 1;);
                i -= 1;
            );
        )
        : (nnote == RHL) ? (                                        // note OFF bottom note, not last voice, scan up for new low RHL
            i = nnote;
            while (!done && i < 128)
            (
                (KeyChannels[i] == RHC) ? (RHL = i; done = 1;);
                i += 1;
            );
        ); // end (nnote == RHL) ?
    ); // end "else it's a note OFF"
);

// driver function
function recalc_bounds(nnote, on_off) (
    (nnote <= SP) ? recalc_bounds_LH(nnote, on_off) : recalc_bounds_RH(nnote, on_off);
);


// Consider moving the split point. Parameter nnote is the last note played
function try_move_SP(nnote) (

/*
    debug_nnote = nnote;
    debug_pass1 = debug_pass2 = debug_pass3 = 0;
    debug_enter0 = debug_enter1 = 0;

//  tentative_hand = (nnote > SP);                          // hand that would be if SP doesn't get moved
//  
//  (tentative_hand != lasthand--) ? (                        // new hand doesn't move SP
//      0;
//  ) :
    (lasthand-- == 0 && debug_enter0 = 1) ? (                 // left hand last? Try to extend it up
        ((nnote > SP)
        && (nnote - SP <= MAXUP ? debug_pass1 = 1)
        && (nnote - lastLH <= RHL - nnote ? debug_pass2 = 1)) ?
            (SP = nnote);
    ): // end (lasthand-- == 0) ?
    (lasthand-- == 1 && debug_enter1 = 1) ? (                 // right hand last? Try to extend it down
        ((nnote < SP)
        && (SP - nnote <= MAXDOWN ? debug_pass1 = 1)
        && (lastRH - nnote <= nnote - LHH ? debug_pass2 = 1)) ?
            (SP = nnote);
    ); // end (tentative_hand != lasthand--) ? - and all chained else's
);
*/
    debug_nnote = nnote;
    (nnote > SP && nnote - lastLH <= lastRH - nnote && nnote - ISP <= MAXUP) ? SP = nnote;
    (nnote <= SP && lastRH - nnote <= nnote - lastLH && ISP - nnote <= MAXDOWN) ? SP = nnote - 1;
);


@slider
// Make channels 0-based (0-15)
LHC = slider1 - 1;
RHC = slider6 - 1;

// Avoid same channel for both hands
(slider1 == slider6) ? slider1 = (slider6  % 16) + 1;

// split point belongs to LH
SP = ISP;                                           // Reset split point


@block

while (midirecv(offset, msg1, msg2, msg3))
(
    status = msg1 & 0xF0;
    channel = msg1 & 0x0F;
    
    is_not_note(status) ? (
        midisend(offset, msg1, msg2, msg3);                         // non-note events go through immediately
        // TODO - other msgs to both hands? filter?
    ):
    (                                                               // else, it's a note
        is_noteoff(status, msg3) ? (                                // note off: just put it out with the right channel
            channel_out = KeyChannels[msg2];
            note_out = KeyNotes[msg2];
            midisend(offset, status | channel_out, note_out, msg3);
            KeyNotes[msg2] = -1;                                    // key not playing anymore: no associated note
            KeyChannels[msg2] = -1;                                 // key not playing anymore: no associated channel
            (msg2 <= SP ? LHV : RHV) -= 1;                          // remember one less voice sounding in current hand
            lasthandoff = (msg2 <= SP) ? 0 : 1;						// remember last hand that played a note off
            recalc_bounds(msg2, 0);                                 // recalculate hand boundaries if deleting notes
        );
        is_noteon(status, msg3) ? (                                 // else, it's note on
            // move the SP if appropriate, will decide which hand gets to play the key
            
            try_move_SP(msg2);
            


            // put the note out on the appropriate channel
            (msg2 <= SP) ? (                                        // Left hand?
                note_out = msg2 + LHT;                              // transpose if needed
                channel_out = LHC;
                lastLH = msg2;                                      // remember the last note
                LHV += 1;                                           // remember one more voice sounding in LH
            ) : (
                note_out = msg2 + RHT;                              // transpose if needed
                channel_out = RHC;
                lastRH = msg2;                                      // remember the last note
                RHV += 1;                                           // remember one more voice sounding in RH
            );
            midisend(offset, (status | channel_out), note_out, msg3);
            KeyNotes[msg2] = note_out;                              // remember which note it was for this key
            KeyChannels[msg2] = channel_out;                        // remember which channel it was for this key
            lasthandon = (msg2 <= SP) ? 0 : 1;						// remember last hand that played a note on
            recalc_bounds(msg2, 1);                                 // recalculate hand boundaries if adding notes
        ); // end is_noteon() ?
    ); // end "else, it's a note"
); // end while (midirecv())

Last edited by juan_r; 05-18-2022 at 06:20 PM. Reason: Added usage notes and parameter tweaking hint
juan_r is offline   Reply With Quote
Old 05-18-2022, 06:23 PM   #9
juan_r
Human being with feelings
 
juan_r's Avatar
 
Join Date: Oct 2019
Posts: 1,075
Default

Quote:
Originally Posted by jrk View Post
No, it needn't move the threshold more if (say) a note is an octave below the LH centre than when a semitone below. Also, it needn't move the threshold (split point) much at a time. Equally, the LH branch needn't move as much as the RH.

But it sounds like you've figured out a way of doing what you want. Thank you for the stimulating question.
I'm giving more thought to your suggestion. It's right: the threshold (split point, SP in my parlance) needn't move as quickly as the hand center. This idea of yours might work. There's the matter of note offs to consider: appropriate action should be taken to recalculate the centers when a note is released.
juan_r is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 11:38 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.