I know nothing about MADI.
According to this, Reaper supports up to 512 channels of input and output
https://www.reaper.fm/about.php
Each *track* supports up to 64 audio channels.
Perhaps Schwa or someone would like to chime in...
I'd see it as an issue of setting audio device options correctly in Reaper Preferences. In particular, setting output channels to 1 thru 96. Could be off base here...
I suppose the Master Track itself only can send 64 channels to the Monitor FX, but there's nothing keeping you from outputing to hardware from other tracks as well.