You may be interested in the CELT codec:
http://www.celt-codec.org/ and the recent development update at
http://people.xiph.org/~xiphmont/demo/celt/demo.html
While NINJAM doesn't depend on getting the absolutely minimum codec/network delay, shaving a few MS off might be handy for keeping under one measure lag at higher BPMs.
API wise it's easier to use than Vorbis. (The reference implementation has a simple one frame in one frame out interface).
Cheers,