[wwwmaze]

Originally Posted by Justin

YV12 version:
Quick note of explanation:

In YV12 mode, the image is processed with 4 pixels at a time (apologies for inaccuracies in terminology):

• y1 = top row, left luminance
• y2 = top row, right luminance
• y3 = bottom row, left luminance
• y4 = bottom row, right luminance
• u = U for 4 pixels
• v = V for 4 pixels

U/V are sort of a vector defining the color and saturation, roughly speaking (not exactly), with values of 128 meaning centered (zero). So their filter states need to get initialized to 128.

So this filter runs the two Y rows as separate filters (avoiding some extra shuffling), and U/V filters separate

Thanks Justin, that's a great explanation. Everything is crystal clear now.

Originally Posted by Justin

The U/V filters probably should have different coefficients since they are running at half the rate

Thanks for the YV12 blur! I see how that could be causing problems. I'm no expert so I need to (and going to) read into it.

Originally Posted by Justin

Both this filter and the simpler one-pole filter have some weird effects on the color/saturation. It might actually make more sense to convert the U/V values to polar coordinates and filter that, but that would be excruciatingly slow.

Since u and v are set to 128 (=zero saturation) at the border and the effect increases at higher sigmas I have a strong feeling that this may be caused by the problem above. I'll investigate further.