|
Buggy VirtualDub filters
August 16, 2012
The same day I was dealing with VDFilter issue described in previous post
I ran Video Enhancer (which also uses VDFilter), picked a random VirtualDub
plugin and tried to process one file. Suddenly I've got a message telling me
about an exception arised inside that VirtualDub plugin. Most VD plugins have
been here for a while and are known for thair speed, stability and high quality,
so I immediately decided the problem was in VDFilter, our DirectShow wrapper
for those plugins. I started a debug session to find what caused the exception
and luckily the source code for that plugin was available on its web page.
The plugin is called Flip 1.1 (ef_flip.vdf from the big collection of filters by
Emiliano Ferrari). To my surprise however I've found the bug quite fast and it was
not in VDFilter, this time it was in the plugin itself. Source code of the main
routine is pretty short:
void Rotate180 (Pixel32 *dst,Pixel32 *tmp,const int pitch,const int w,const int h,
const FilterFunctions *ff)
{
Pixel32 *a,*b;
int i;
a= dst;
b= dst + (h*pitch);
int alf = h/2;
if (h&1) alf++;
for (int j=0; j<alf; j++)
{
_memcpy (tmp,a,w*sizeof(Pixel32)); // tmp= a;
a+= w;
for (i=0; i<w; i++) *a--= *b++; // a=b
for (i=0; i<w; i++) *b--= *tmp++; // b=tmp
tmp-= w;
a+= pitch;
b-= pitch;
}
if (!(h&1)) // even lines
{
_memcpy (tmp,a,w*sizeof(Pixel32));
a+= w;
for (i=0; i<w; i++) *a--= *tmp++;
}
}
Take a look at the two red lines. h is the image height
and pitch here is equal to the number of Pixel32 values in one row of image, so
h*pitch is the number of pixels in whole image. At first pointer b points
to dst + (h*pitch), i.e. the first byte outside the image buffer! And then
in the loop it's being read and then incremented to point even farther from the end
of the buffer. It didn't cause crashes in VirtualDub because for some reason memory
after the image buffer always belonged to the application, however it did read garbage
there so the first line of "rotated" image should contain garbage after applying this
filter. In case of VDFilter and its use in Video Enhancer one line of memory after the
image buffer not always completely belonged to the program, so for small frames it
worked ok but with a larger frame the b pointer walked too far and caused
a segmentation fault which was caught as an exception and caused our wrapper to
show the message box.
It may be the case that author assumed pitch to be a negative value, however
this assumption doesn't look correct. Here's a quote from VirtualDub Plugin SDK:
"Bitmaps can be stored top-down as well as bottom-up. The pitch value value is positive
if the image is stored bottom-up in memory and negative if the image is stored top-down.
This is only permitted if the filter declares support for flexible formats by returning
FILTERPARAM_SUPPORTS_ALTFORMATS from paramProc; otherwise, the host ensures that the
filter receives a bottom-up orientation with a positive pitch, flipping the bitmap
beforehand if necessary."
Lessons learned: 1) some VirtualDub plugins, even very simple ones, may contain bugs.
2) pointer arithmetic requires a lot of attention to be used correctly.
tags: video_enhancer
| 
RSS
