Image processing optimization

Question

I'm using Winforms C# .NET 3.5. I'm getting frames, and this is how I handle them:

    delegate void videoStream_NewFrameDelegate(object sender, NewFrameEventArgs eventArgs);
    public void videoStream_NewFrame(object sender, NewFrameEventArgs eventArgs)
    {
        if (ready)
        {
            if (this.InvokeRequired)
            {
                videoStream_NewFrameDelegate del = new videoStream_NewFrameDelegate(videoStream_NewFrame);
                this.Invoke(del, new object[] {sender, eventArgs} );
            }
            else
            {
                Rectangle rc = ClientRectangle;
                Bitmap bmp = new Bitmap(rc.Width, rc.Height);
                Graphics g = Graphics.FromImage((Image)bmp);
                g.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBilinear;
                g.DrawImage((Bitmap)eventArgs.Frame, rc.X+10, rc.Y+10, rc.Width-20, rc.Height-20);

                g.Dispose();
                this.Image = (Image)bmp;
            }
        }
    }

Is there any way to optimize or improve performance?

I'm getting low performance when my image is 320x240 and I'm stretching it to 1280x1024, But the same image on 640x480 and stretching to 1280x1024 wont get much loss in performance.

I tried to use WPF and still same performance loss. That is weird because WPF is supposed to use DirectX and be fast in image processing.

Here is my WPF code:

    delegate void videoStream_NewFrameDelegate(object sender, NewFrameEventArgs eventArgs);
    public void videoStream_NewFrame(object sender, NewFrameEventArgs eventArgs)
    {
        if (ready)
        {
            if (!this.Frame.Dispatcher.CheckAccess())
            {
                videoStream_NewFrameDelegate del = new videoStream_NewFrameDelegate(videoStream_NewFrame);
                this.Frame.Dispatcher.Invoke(del, new object[] { sender, eventArgs });
            }
            else
            {
                Bitmap bmp = (Bitmap)eventArgs.Frame.Clone();
                IntPtr hBitmap = bmp.GetHbitmap();
                BitmapSource img = Imaging.CreateBitmapSourceFromHBitmap(hBitmap, IntPtr.Zero, Int32Rect.Empty, BitmapSizeOptions.FromEmptyOptions());
                bmp.Dispose();
                GC.Collect();

                this.Frame.Source = img;
            }
        }
    }

Answer 1

I tried to use WPF and still same preformence lose.. that is weird because WPF suppose to use DirectX and be fast in image proccessing.

Except, in the WPF version, you are cloning a Bitmap (which has nothing to do with WPF) and then calling Imaging.CreateBitmapFromHBitmap, which eventually resolves to a call to:

[DllImport("WindowsCodecs.dll", EntryPoint="IWICImagingFactory_CreateBitmapFromHBITMAP_Proxy")]
internal static extern int CreateBitmapFromHBITMAP(IntPtr THIS_PTR, IntPtr hBitmap, IntPtr hPalette, WICBitmapAlphaChannelOption options, out BitmapSourceSafeMILHandle ppIBitmap);

Now I don't know what that method is doing, but it may very well be copying the data again, but perhaps not. Either way, your WPF method is performing at least one clone of the entire image and your WinForms version is not.

That's not really the meat of it though. You haven't even posted benchmark results, so you need to do that first before assuming any one part of code is "slow", and you will have a hard time optimizing until you have that answer. You're two versions aren't even identical in output; the WinForms version performs HighQualityBilinear interpolation, which is certainly going to take a significant amount of time.

You need to tell us what you requirements are. I ran a quick test with the following code using a 320x240 size image, and then again at 640x480:

public Form1()
{
    InitializeComponent();
}   

private TimeSpan RunTest( int sampleSize )
{
    var sw = Stopwatch.StartNew();
    var baseImg = new Bitmap( @"C:\TestImage.jpg" );
    Bitmap bmp = null;
    for( int i = 0; i < sampleSize; ++i )
    {
        var rect = picBox.ClientRectangle;
        bmp = new Bitmap( rect.Width, rect.Height );
        using( Graphics g = Graphics.FromImage( bmp ) )
        {
            g.InterpolationMode = InterpolationMode.HighQualityBilinear;
            g.DrawImage( baseImg, rect );
        }

        picBox.Image = bmp;
            // the call to Invalidate() that you have is not needed.
    }

    sw.Stop();
    return TimeSpan.FromMilliseconds( sw.ElapsedMilliseconds );
}

private void picBox_Click( object sender, EventArgs e )
{
    int iterations = 100;
    var time = RunTest( iterations );
    MessageBox.Show( String.Format( 
        "Elapsed: {0}, Average: {1}", 
        time.TotalMilliseconds, 
        time.TotalMilliseconds / iterations ) );
}

I then performed another test using the default InterpolationMode. Under a release build that gave me the following results:

320x240 1. High quality interpolation: Total: 3122ms, Average: 31.22ms, FPS: 32 2. Default interpolation: Total: 2165ms, Average: 21.65ms, FPS: 46

640x480 1. High quality interpolation: Total: 3963ms, Average: 39.63ms, FPS: 25 2. Default interpolation: Total: 2256ms, Average: 22.56ms, FPS: 44

Are you saying that 32 frames per second is not good enough for your application? Is 42 ok and is it alright to remove the high quality rendering? You need to provide more details. You can't expect us to optimize your code with no target requirement in mind. Also, as you can see, your claim that the performance is better with large images does not appear to be true, though the difference without high quality bilinear rendering is negligible.

If you can forego the Bilinear interpolation you can simply set the StretchMode of the PictureBox to Stretch and just set the Image property to the Frame property of the event args object and be done with it. This will be significantly faster.