# Image processing optimization

I'm using Winforms C# .NET 3.5. I'm getting frames, and this is how I handle them:

    delegate void videoStream_NewFrameDelegate(object sender, NewFrameEventArgs eventArgs);
public void videoStream_NewFrame(object sender, NewFrameEventArgs eventArgs)
{
{
if (this.InvokeRequired)
{
videoStream_NewFrameDelegate del = new videoStream_NewFrameDelegate(videoStream_NewFrame);
this.Invoke(del, new object[] {sender, eventArgs} );
}
else
{
Rectangle rc = ClientRectangle;
Bitmap bmp = new Bitmap(rc.Width, rc.Height);
Graphics g = Graphics.FromImage((Image)bmp);
g.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBilinear;
g.DrawImage((Bitmap)eventArgs.Frame, rc.X+10, rc.Y+10, rc.Width-20, rc.Height-20);

g.Dispose();
this.Image = (Image)bmp;
}
}
}


Is there any way to optimize or improve performance?

I'm getting low performance when my image is 320x240 and I'm stretching it to 1280x1024, But the same image on 640x480 and stretching to 1280x1024 wont get much loss in performance.

I tried to use WPF and still same performance loss. That is weird because WPF is supposed to use DirectX and be fast in image processing.

Here is my WPF code:

    delegate void videoStream_NewFrameDelegate(object sender, NewFrameEventArgs eventArgs);
public void videoStream_NewFrame(object sender, NewFrameEventArgs eventArgs)
{
{
if (!this.Frame.Dispatcher.CheckAccess())
{
videoStream_NewFrameDelegate del = new videoStream_NewFrameDelegate(videoStream_NewFrame);
this.Frame.Dispatcher.Invoke(del, new object[] { sender, eventArgs });
}
else
{
Bitmap bmp = (Bitmap)eventArgs.Frame.Clone();
IntPtr hBitmap = bmp.GetHbitmap();
BitmapSource img = Imaging.CreateBitmapSourceFromHBitmap(hBitmap, IntPtr.Zero, Int32Rect.Empty, BitmapSizeOptions.FromEmptyOptions());
bmp.Dispose();
GC.Collect();

this.Frame.Source = img;
}
}
}

-
Use Managed D2D withing WPF window –  Artur Mustafin Sep 21 '11 at 4:33
@Artur You got an example ? –  Danpe Sep 21 '11 at 10:36

I tried to use WPF and still same preformence lose.. that is weird because WPF suppose to use DirectX and be fast in image proccessing.

Except, in the WPF version, you are cloning a Bitmap (which has nothing to do with WPF) and then calling Imaging.CreateBitmapFromHBitmap, which eventually resolves to a call to:

[DllImport("WindowsCodecs.dll", EntryPoint="IWICImagingFactory_CreateBitmapFromHBITMAP_Proxy")]
internal static extern int CreateBitmapFromHBITMAP(IntPtr THIS_PTR, IntPtr hBitmap, IntPtr hPalette, WICBitmapAlphaChannelOption options, out BitmapSourceSafeMILHandle ppIBitmap);


Now I don't know what that method is doing, but it may very well be copying the data again, but perhaps not. Either way, your WPF method is performing at least one clone of the entire image and your WinForms version is not.

That's not really the meat of it though. You haven't even posted benchmark results, so you need to do that first before assuming any one part of code is "slow", and you will have a hard time optimizing until you have that answer. You're two versions aren't even identical in output; the WinForms version performs HighQualityBilinear interpolation, which is certainly going to take a significant amount of time.

You need to tell us what you requirements are. I ran a quick test with the following code using a 320x240 size image, and then again at 640x480:

public Form1()
{
InitializeComponent();
}

private TimeSpan RunTest( int sampleSize )
{
var sw = Stopwatch.StartNew();
var baseImg = new Bitmap( @"C:\TestImage.jpg" );
Bitmap bmp = null;
for( int i = 0; i < sampleSize; ++i )
{
var rect = picBox.ClientRectangle;
bmp = new Bitmap( rect.Width, rect.Height );
using( Graphics g = Graphics.FromImage( bmp ) )
{
g.InterpolationMode = InterpolationMode.HighQualityBilinear;
g.DrawImage( baseImg, rect );
}

picBox.Image = bmp;
// the call to Invalidate() that you have is not needed.
}

sw.Stop();
return TimeSpan.FromMilliseconds( sw.ElapsedMilliseconds );
}

private void picBox_Click( object sender, EventArgs e )
{
int iterations = 100;
var time = RunTest( iterations );
MessageBox.Show( String.Format(
"Elapsed: {0}, Average: {1}",
time.TotalMilliseconds,
time.TotalMilliseconds / iterations ) );
}


I then performed another test using the default InterpolationMode. Under a release build that gave me the following results:

320x240 1. High quality interpolation: Total: 3122ms, Average: 31.22ms, FPS: 32 2. Default interpolation: Total: 2165ms, Average: 21.65ms, FPS: 46

640x480 1. High quality interpolation: Total: 3963ms, Average: 39.63ms, FPS: 25 2. Default interpolation: Total: 2256ms, Average: 22.56ms, FPS: 44

Are you saying that 32 frames per second is not good enough for your application? Is 42 ok and is it alright to remove the high quality rendering? You need to provide more details. You can't expect us to optimize your code with no target requirement in mind. Also, as you can see, your claim that the performance is better with large images does not appear to be true, though the difference without high quality bilinear rendering is negligible.

If you can forego the Bilinear interpolation you can simply set the StretchMode of the PictureBox to Stretch and just set the Image property to the Frame property of the event args object and be done with it. This will be significantly faster.

-
Hey! thx alot for the awosome answer ! i added some test to my questions. My event is getting called when a WebCam frame is arrived. I tried what you said with the stretch mode but i get Empty Image.. this.Image = (Bitmap)eventArgs.Frame; I dont know why but when i'm taking the 320x240 picture and stretching it in a full screen PictureBox i get Long delay, but the tests you you do shows only 4500ms on full screen, and 3300 on normal size.. but in the eye it looks like 3 seconds delay.. –  Danpe Sep 14 '11 at 18:41
@Danpe: My timings may have been a bit misleading; they don't take into account the time that the PictureBox takes to redraw itself, which is bound to be significant. –  Ed S. Sep 19 '11 at 20:38
@Ed S.: your "sample" is not useful, and do not provide significant performance improvements compared to the original post. Is any case, it will have almost the same timings, because it usese the same managed function calls –  Artur Mustafin Sep 20 '11 at 1:44
@Ed S.: You did not turn .SizeMode to StretchImage in a PictureBoxControl. I have an 1080p image in a 200x200 box, and my code as is is 75% faster, than your "sample", with timings, inclding .Image assigments. So your code sucks –  Artur Mustafin Sep 20 '11 at 2:04
@Artur: Well put sir, you are obviously extremely mature and talented. Also, my code was posted in an attempt to get some baseline timings for the OP's code and then I asked some more questions. Nowhere did I say that it was an improvement. Please refrain from crying over my downvote on your response. Go grab a few rep points and downvote me back if it makes you feel better. –  Ed S. Sep 20 '11 at 2:09