The thumbwheel on my Logi MX Master functions as scroll left/right in Reaper by default, which is great. But is there a way to have shift+thumbwheel function as scrub?
Not natively. You could use a MIDI encoder/knob. But if your mousewheel is also a button you can press, you can assign middle drag to scrub in the mouse modifiers.