Or due to the impulse being loaded while a buffer is being convolved. Loading the impulse is not an atomic action, so unless you're guarding with some sort of lock you could be convolving with some random percentage of new impulse + remainder of old impulse.
Of course, locking access to the impulse has its own set of problems...including dropped buffers like was mentioned earlier.
Or, maybe it's late in the day and I'm tired and I'm totally off base.