Select notes before / after edit cursor

[_Stevie_]

This is another script I'm trying to adapt to Get/SetAllEvts.
I really thought, this would be easier, but after 3 days of going back and forth, I still haven't found the solution.

I know why the script is not working, but I have no idea how to solve it.
The issue is a note-off that crosses the edit cursor. In that case, the note-off will be selected, whereas the corresponding note-on won't. As result, the note will be stretched until the end of the take.

I need to get the start and end point of a note, to see if the note start lies BEFORE the edit cursor. If that's the case, the corresponding note-off shall not be selected, because it's crossing the edit cursor.
But to do this, I need to get the right pair of note-on and note-off. This is like a puzzle... Any idea how to achieve this?

That's my (shortened to essence) code:

while stringPos < MIDIlen-12 do
	offset, flags, msg, stringPos = string.unpack("i4Bs4", MIDIstring, stringPos)
	sumOffset = sumOffset+offset -- add all event offsets to get next start position of event on each iteration
	eventStart = itemStartPPQ+sumOffset -- calculate event start position based on item start position
	if #msg == 3
	and (msg:byte(1)>>4) == 9 and eventStart >= cursorPositionPPQ 
	or (msg:byte(1)>>4) == 8 and eventStart-offset > cursorPositionPPQ
	then
		if flags == 0 or flags == 1 then flags = 1
		else flags = 3
		end
	end
	table.insert(tableEvents, string.pack("i4Bs4", offset, flags, msg))
end

[FnA]

How's it going? I guess I'd do something somewhat similar to what I did for the join notes script with a table with each note that has a reference to the index in the other table where the event needing changing is.

[_Stevie_]

Still haven't made any progress
But your approach sounds like an idea. Do you have the link to your script handy?

[FnA]

It's in ReaTeam Scripts/MIDI Editor.

There's a related script here, which was the discussion thread for the one that got put in ReaPack. It has, I think, the same type of method used, but is limited to a certain time frame. The first version of the Join script used the old MIDI_SetNote methods, but didn't work with overlapping notes.

https://forum.cockos.com/showthread.php?t=143366

You won't be able to use it (exactly) as is. More of a conceptual similarity kind of thing. I think I see a couple optimizations now that could be done, so no claim that there is no better way. I was hoping someone would show up with something slicker, but I'm pretty sure you can make something work like this.

[_Stevie_]

Thanks FnA! That sounds like a good starting point. Will try to understand the code. Looks not very easy with all the tables

[Meo-Ada Mespotine]

Have you tried looking into the MediaItemStateChunks for inspiration? Maybe you can find start and endpoint of a note in there more easily than with the API....

The MIDI-notes are placed in the SOURCE-part of the StateChunk.

[FnA]

The immediately preceding note on may not be the one that belongs to a particular note off. Overlapping notes, in other words.

edit. Use of SetAllEvents will preserve overlapping notes in a way where Note off 2 goes with Note on 2; 3 with 3, etc. It's arbitrary treatment of what can exist in a more complex way in the MIDI Editor. (Probably the best arbitrary choice, I suppose) You can see how it will do that even in a script that just gets and sets the same string. I can't remember how I set up the experiment, but I managed to effectively swap note ons, because of this, in two or more notes that started at the same position. Noticeable symptom was that it exchanged the velocities. Other procedures in Reaper will do this kind of re-sorting too. Undo. Changing active item, some other thing or two. It's been a while since I experimented. I think it's best to just go with 1-1,2-2,3-3, etc. But you have to consider each of the "flags" as a different flavor that can exist separately too.

The Glue/Heal script probably has a lot of unnecessary distractions in it, relative to your case.

[juliansader]

In my scripts, I use a table to store the info of the last note-on in each channel, pitch and flag, so that later note-offs can be matched with the correct note-on.

You can store the channel, pitch and flags info as separate subtables:

tNoteons[chan][pitch][flags] = {whatever info}

or you can combine the channel, pitch and flag numbers into a single number:

noteID = (chan<<9) | (pitch<<2) | flags
tNoteons[noteID] = {...

This is an excerpt from one of my scripts:

-- While parsing the MIDI string, the indices of the last note-ons for each channel/pitch/flag must be temporarily stored until a matching note-off is encountered. 
local runningNotes = {}
for channel = 0, 15 do
    runningNotes[channel] = {}
    for pitch = 0, 127 do
        runningNotes[channel][pitch] = {} -- Each pitch will have flags as keys.
    end
end

...

if eventType == 9 and msg:byte(3) ~= 0 then -- Note-ons
            local channel = msg:byte(1)&0x0F
            local pitch   = msg:byte(2)
            if runningNotes[channel][pitch][flags] then
                reaper.ShowMessageBox("The script encountered overlapping notes.\n\nSuch notes are not valid MIDI, and can not be parsed.", "ERROR", 0)
                return false
            else
                tNotes[#tNotes+1] = {noteOnIndex = e}
                runningNotes[channel][pitch][flags] = #tNotes
            end
        
        elseif eventType == 8 or (eventType == 9 and msg:byte(3) == 0) then
            local channel = msg:byte(1)&0x0F
            local pitch   = msg:byte(2)
            local lastNoteOnIndex = runningNotes[channel][pitch][flags]
            if lastNoteOnIndex then
                tNotes[lastNoteOnIndex].noteOffIndex = e
                runningNotes[channel][pitch][flags] = nil
end

[FnA]

Ah, yeah. I guess that would be the safest thing to do. Shouldn't mess up any thing if it exits without Setting AllEvents.

[_Stevie_]

Thanks guys! I'll get back to scripting ASAP. Need to get some work done and then I'm back to solve this riddle :P

[FnA]

Hm. There's two actions for selecting notes in/starting in time selection. (That would be multi item capable, I think)

but not for the inline editor and I see your current script is also available in Main, for selected items, I suppose, so...

[_Stevie_]

Yep, most of my MIDI scripts work in the MIDI editor, inline and main, because it's not always needed to open the MIDI editor for certain tasks.

[FnA]

So, it appears that there's no function to provide the take that an inline action came from? Maybe a good API request?

Unrelated- Any one know if it's possible to tell if the regular midi editor has focus or any other particular window for that matter?

The time selection thing I mention seems to be the only way to preserve complicated note overlaps that can exist in the midi editor. It would possibly make a graphics glitch or flash, but otherwise, you have to deal with the arbitrary re-sort pattern I mention or exit, as hinted at in Julian's extract. I ran the action a bit just now, but didn't test much more than that.

[Thonex]

Originally Posted by _Stevie_

Yep, most of my MIDI scripts work in the MIDI editor, inline and main, because it's not always needed to open the MIDI editor for certain tasks.

Thank you!

I try to avoid opening the Midi editor as much as I can. In Nuendo, I "knew" what was being applied to my MIDI tracks because I had worked with it so long. In Reaper... I don't feel comfortable/confident enough with the MIDI commands to do it without opening the MIDI editor to check my work. That's more because of being new to Reaper than it's MIDI shortcomings... just haven't used Reaper with MIDI enough.

[juliansader]

Originally Posted by FnA

So, it appears that there's no function to provide the take that an inline action came from? Maybe a good API request?

I have indeed submitted such as FR: ReaScript: API functions to get and set focus on an item in the arrange view.

If the mouse is over the item, Breeder's SWS functions can help.

Unrelated- Any one know if it's possible to tell if the regular midi editor has focus or any other particular window for that matter?

I have submitted a large number of window-info-related functions for SWS, including Window_GetFocus. However, I don't know how long it will take for the next SWS update to be released. I am considering releasing the functions as a mini extension via ReaPack instead, since the extension will only be of interest to scripters. (Unfortunately, I can only build for Windows.)

[_Stevie_]

Originally Posted by Thonex

Thank you!

I try to avoid opening the Midi editor as much as I can. In Nuendo, I "knew" what was being applied to my MIDI tracks because I had worked with it so long. In Reaper... I don't feel comfortable/confident enough with the MIDI commands to do it without opening the MIDI editor to check my work. That's more because of being new to Reaper than it's MIDI shortcomings... just haven't used Reaper with MIDI enough.

I hear ya, in the past I have created so many LE presets in Cubase... Most of them, I recreated in Reaper. Still some left, though!

The good thing with LE is: it applies the MIDI edits no matter where you are.
Arrangement, MIDI editor... it even changes multiple parts at once, when you have them selected. I got so used to that behaviour, that I included it in my scripts (where it makes sense).

[Thonex]

Originally Posted by _Stevie_

I hear ya, in the past I have created so many LE presets in Cubase... Most of them, I recreated in Reaper. Still some left, though!

The good thing with LE is: it applies the MIDI edits no matter where you are.
Arrangement, MIDI editor... it even changes multiple parts at once, when you have them selected. I got so used to that behaviour, that I included it in my scripts (where it makes sense).

We should start a "Moving from Cubendo to Reaper" script base

[_Stevie_]

Originally Posted by Thonex

We should start a "Moving from Cubendo to Reaper" script base

Count me in!

[FnA]

@juliansader
I don't know how much I can contribute to testing that you can't do yourself, but do you have a build already? I use Windows 7x64. At any rate, I'd like to read more about the proposed functions. Thanks for the large amount of work you do, and help you give.

[juliansader]

Originally Posted by FnA

@juliansader
I don't know how much I can contribute to testing that you can't do yourself, but do you have a build already? I use Windows 7x64. At any rate, I'd like to read more about the proposed functions. Thanks for the large amount of work you do, and help you give.

It would be great if you could test some of the new functions!

I have uploaded the x64 .dll file to the stash. Simply replace your existing SWS dll in the C:/Program Files/REAPER (x64)/plugins/ folder with the new file. In the API help, look for the new Window_, Mouse_ and MIDIEditor_ functions.

I can upload code snippets to demonstrate some simple applications, but in the meantime, here are some FRs solved by the new functions:
* Converting "reaper object" pointers to strings and back
* MIDIEditor_GetSetting_int: Return window client area coordinate
* Reascript: MIDIEditor_CountEditors and MIDIEditor_GetEditor
* Allow script windows to set/get focus
* Exposure of mouse state-functions for non gfx.ini-created windows
* Use scripts with "drag" mouse modifiers

[FnA]

Originally Posted by juliansader

It would be great if you could test some of the new functions!

wow. That's a lot of functions. I bet a lot of good will come out of these. Quite a lot of it is over my head. I just tried a bunch to see how they work (kind of) and didn't get any obvious problems. Some that were probably my own sloppiness.

Originally Posted by juliansader

I can upload code snippets to demonstrate some simple applications,

That would be good. Can't wait to see what YOU have in mind for these. Maybe there should be a separate thread, in developer forum perhaps. I guess I will look into solutions to two threads I have on the forums. Mouse state function should help with time/loop selection undo/redo. Get/Set focus should help with a couple dock/undock actions affected by "Opening/closing docked MIDI Editor can cause irretrievable loss of arrange view vertical zoom" bug. I had some luck with save/restore arrange, but some issues with focus. (I cant resist: I saw GetScrollinfo and immediately thought "oh boy! is there SetScroll?) I think a monitor fx hide/show script has an issue like that too. Also maybe: save/restore FX window slots might be something easier/cleaner/less loopy with these. Well, like I said, I'm sure a lot of good will come of it.

[_Stevie_]

Julian, that's amazing! Is there a function that has the potential that could solve this FR?
https://forum.cockos.com/showthread.php?t=209321

[juliansader]

Originally Posted by _Stevie_

Julian, that's amazing! Is there a function that has the potential that could solve this FR?
https://forum.cockos.com/showthread.php?t=209321

Unfortunately I have not (yet) been able to find a cross-platform method for intercepting keystrokes. (Perhaps experts such as cfillion may know a way.) Strangely, mouse events are passed to the foreground window's "message queue", but keystrokes appear not to be.

If you are on Windows, AutoHotKey can intercept keystrokes and channel them to specified windows.

[FnA]

The functions can, at least, return focus to previous window when mouse button is released. In a deferred script.

[_Stevie_]

Okay guys, I again have some time for coding. I will try to wrap my head around this.

[_Stevie_]

You know what's really annoying? Taking a 3 weeks break from scripting and you don't understand anything at all anymore... that's so frustrating.

[Lokasenna]

It's difficult, but I try to always follow one important rule for writing code:

Remember that You In Three Months is going to be reading it, and name/refactor/comment accordingly.

[_Stevie_]

Yeah absolutely, that's why I comment like crazy. But things that are obvious to me at the time of coding, might completely convoluted when I get to coding!

So, I really need to comment more extesively...
Plus, Get/SetAllEvts is really something that's not easy to handle for my brain.

[FnA]

Don't feel too bad. There's quite a bit of code syntax to get a handle on. And then, the further you get into positioning things, the harder it gets. I'm pretty much discouraged from posting midi scripts in that regard myself. Reaper can throw you some curveballs. Imagine trying to do something based on project grid in ignore project tempo item which may be stretched, have start offset, not begin on grid, cross gradual tempo changes. Sucks.

[Lokasenna]

Originally Posted by _Stevie_

Yeah absolutely, that's why I comment like crazy. But things that are obvious to me at the time of coding, might completely convoluted when I get to coding!

Are more comments the answer, or more readable code?

Naming functions/variables so that they're obvious, breaking giant functions into a bunch of smaller clearly-named steps, organizing the code by task or something... there's a lot you can do that makes comments less necessary.

[Thonex]

Originally Posted by Lokasenna

Are more comments the answer, or more readable code?

Naming functions/variables so that they're obvious, breaking giant functions into a bunch of smaller clearly-named steps, organizing the code by task or something... there's a lot you can do that makes comments less necessary.

Readable code is key for me. The ONLY time I will ever use a single character variable is for counters. And even then, when it's complex code I'll only use (for example) "i" as a prefix denoting a counter... like i_Prev_Items.

There is a long debate between commenting and verbose variables. Ideally I do both. But if it's a 20 line piece of code without many layers of abstraction, then I comment less.

But the most difficult thing for me is when I'm in the middle of coding a complex algo, and my wife comes in and asks me to take out the trash... completely derailing my train of thought LOL. It will sometime take me 15 minutes to get back into the headspace I was in... and even then... sometimes I will have lost that kernel of viable solution LOL

[Lokasenna]

If your variable names are getting really long then there's probably another layer of "break it into smaller functions/objects" in there.

One piece of advice I see a lot is that, if you need to comment a function to explain what it's doing, you could probably rename the function instead.

Originally Posted by Thonex

But the most difficult thing for me is when I'm in the middle of coding a complex algo, and my wife comes in and asks me to take out the trash... completely derailing my train of thought LOL.

I write a lot of my scripts out as comments first and then fill in the gaps, and anything complicated probably starts off on paper.

[_Stevie_]

Originally Posted by FnA

Don't feel too bad. There's quite a bit of code syntax to get a handle on. And then, the further you get into positioning things, the harder it gets. I'm pretty much discouraged from posting midi scripts in that regard myself. Reaper can throw you some curveballs. Imagine trying to do something based on project grid in ignore project tempo item which may be stretched, have start offset, not begin on grid, cross gradual tempo changes. Sucks.

Ah thanks man, that's encouraging, that you are having the same issues.
Of course it's a always a challenge somehow, but if I see all the crazy scripts you guys are pulling off...

Are more comments the answer, or more readable code?

Naming functions/variables so that they're obvious, breaking giant functions into a bunch of smaller clearly-named steps, organizing the code by task or something... there's a lot you can do that makes comments less necessary.

Absolutely, refactoring! But when I check back into my scripts it's like: why the hell did I do that? The comment doesn't help at all here! Back then, it made completely sense, since I was completely into coding, but after a while I forget about these methods I used. I guess the only thing that would help is to record myself talking about it, haha.


Another issue is, that I have so many things at the moment in parallel. So, I don't know where to continue...

- scripts that needs refatctoring
- script ideas, that I have yet to put into an algorithm
- scripts that haven't been finished yet since I took a break at the wrong time (Thonex' trash example! Haha)
- adapting my MIDI scripts to Set/GetAllEvts, like the current one in this thread


And last but not least: the problem of code visualisation. Like: what happens
with the data when it goes thru all the different functions... e.g. keeping track of the whole data and how it is processed. At least for me, it takes a lot of concentration.

[Thonex]

Originally Posted by Lokasenna

I write a lot of my scripts out as comments first and then fill in the gaps, and anything complicated probably starts off on paper.

I like that.

It's like developers that start with a UI design and then code.. as to avoid feature-creep (guilty as charged).

[_Stevie_]

Yep, me too! Very good approach.

I'm now trying to break my algorithm idea down. Could anyone of you have a look, if this could work?
This is the hardest part for me, to think of all the side-effects.

if note-on
   store note-on in table with pitch info

elseif note-off and before cursor
   get note-on with same pitch from table
   set select flag for note-on and note-off  

elseif note-off after cursor 
   get note-on with same pitch from table
   set unselect flag for note-on and note-off

[FnA]

I think you need to specify how you want to approach the overlapping note situation at this time. I think if you choose the exit strategy, it will require less code. But you may have processed several items when you run into an overlap. I don't think it's possible to handle them perfectly otherwise, to the point where a clever bug hunter could not find some undesired behavior to complain about, although you can get pretty close to the way Reaper handles them outside the midi editors. I think the inline editor is the hard one to handle here.

I'm going to go ahead and just try the time selection action I mentioned earlier in a script for the main editor. But I've been confused as to what you really want for these notes that cross the cursor. Do you want the new scripts to behave exactly the same as the old? If not please describe behavior in each script.

[_Stevie_]

That's a good point FnA, I think an error message would suffice for the start.
I personally always try to avoid overlapping notes, even in other DAWs.

Concerning notes crossing the cursor... I think I would ignore those notes and not include them in the selection. So it will behave pretty much the same as the old script, but with the fast SetAllEvts method.

[_Stevie_]

Now, that I think about it, it would be easier to also include the crossing notes...

New algorithm (again, not sure, if it covers everything):

if note-on before cursor then
  select note-on
  store note-on in table with pitch
elseif note-on after cursor then
  unselect note-on
														
elseif note-off after cursor then
  if appropriate note-on before cursor then
    select note-off
  else
    unselect note-off

[FnA]

Originally Posted by _Stevie_

Now, that I think about it, it would be easier to also include the crossing notes...

That is how it works with the select all notes starting in time selection action. This works on all Editable items, which may or may not be desired. I think it might be handy to have as well as one that only does Active item.


So, anyway...concentrate on the "before cursor" script first?

I think Julian's post (8) is a good reference. I think you can get it going with the 16 x 128 (by 2 muted/unmuted?) table for now then reduce that, if possible, later. Maybe 16 by 2, with pitch as key would be good enough? I don't think you need to worry about selected flags. Remember that notes on different channels are ok overlapping, and I'm pretty sure that a muted note can sit anywhere in a nonmuted one and vice versa.

Maybe you can shift gears after the cursor is reached and all notes crossing it are off somehow.

[_Stevie_]

Originally Posted by FnA

So, anyway...concentrate on the "before cursor" script first?

Definitely!

Originally Posted by FnA

I think it might be handy to have as well as one that only does Active item.

It would make it dependent on item selection, e.g. how many items are selected.

Originally Posted by FnA

I think Julian's post (8) is a good reference. I think you can get it going with the 16 x 128 (by 2 muted/unmuted?) table for now then reduce that, if possible, later. Maybe 16 by 2, with pitch as key would be good enough? I don't think you need to worry about selected flags. Remember that notes on different channels are ok overlapping, and I'm pretty sure that a muted note can sit anywhere in a nonmuted one and vice versa.

Yeah, I tried to wrap my head around it, but I don't understand all the table stuff tbh

Originally Posted by FnA

Maybe you can shift gears after the cursor is reached and all notes crossing it are off somehow.

Yep, that's what I got so far. Hence, I haven't included the table condition, yet.
It's only pseudo code.

if msg:len() == 3 then -- if msg consists of 3 bytes (= channel message)
    msg_b1_nib1 = msg:byte(1)>>4 -- save 1st nibble of status byte (contains info about the data type) to msg_b1_nib1, >>4 shifts the channel nibble into oblivion

    -- note on before cursor position? = select
    if msg_b1_nib1 == 9 and eventStart < cursorPositionPPQ then 
        if flags == 0 or flags == 1 then flags = 1 	-- unselected or selected? = select event
        else flags = 3 								-- muted or muted selected? = select muted event
        end
        store note-on in table with pitch
    
    -- note on after cursor position? = unselect 
    elseif msg_b1_nib1 == 9 and eventStart > cursorPositionPPQ then
        if flags == 0 or flags == 1 then flags = 0 	-- unselected or selected? = unselect event
        else flags = 2 								-- muted or muted selected? = unselect muted event
        end

    -- note off after cursor position and note-on before cursor? = select
    elseif msg_b1_nib1 == 8 and eventStart > cursorPositionPPQ then
        if appropriate note-on stored in table then  -- if note on before cursor
            if flags == 0 or flags == 1 then flags = 1	-- unselected or selected? = select event
            else flags = 3 								-- muted or muted selected? = select muted event
            end
        else
            if flags == 0 or flags == 1 then flags = 0	-- unselected or selected? = unselect event
            else flags = 2 								-- muted or muted selected? = unselect muted event
            end
        end
            
    -- if note off after cursor position? = unselect
    elseif msg_b1_nib1 == 8 and eventStart > cursorPositionPPQ then
        if flags == 0 or flags == 1 then flags = 0 	-- unselected or selected? = unselect event
        else flags = 3 								-- muted or muted selected? = select muted event
        end
    end
end