Scanning image pixels

Question

This code checks 2 images' pixels, compares each pixels based on the threshold (<= 7), finds a best fitting watermark image from the system given 6 images, and loads that image next to the cover image.

For a 100*100 watermark image this particular code takes nearly 130 secs to give output.

Please suggest some ways to optimize it regarding execution time.

namespace Watermarking
{
   public partial class Embedding_Pixels_Evaluation : Form
     {
            private Stopwatch sp;


public Embedding_Pixels_Evaluation()
{
    InitializeComponent();
    sp = new Stopwatch();
    button1.Enabled = false;

}

//choose watermark from system
private void button2_Click(object sender, EventArgs e)
{
    sp.Start();
    Boolean imageSize = false;
    Boolean flag = false;

    int[] count_miss = new int[6];
    int[] count_hit = new int[6];

    for (int cnt = 1; cnt <= 6; cnt++)
    {

        //sample watermarks saved in sampleimage folder
        string curpath = Application.StartupPath.ToString() + @"\sampleimage\";
        curpath = curpath + cnt.ToString() + ".png";

        //watermark_img1 = Bitmap.FromFile(curpath);
        //Image<Bgr, byte> watermark_img = new Image<Bgr, byte>(watermark_img1);
        Image<Bgr, byte> watermark_img = new Image<Bgr, byte>(curpath);
        Image<Bgr, byte> cover = new Image<Bgr, byte>((Bitmap)pictureBox1.Image);
        int threshold = 7;
        var blocked = new HashSet<int>();

        if (watermark_img.Height * watermark_img.Width <= cover.Height * cover.Width && cover.Height * cover.Width <= 2 * watermark_img.Height * watermark_img.Width)
        {
            imageSize = true;
            //preserve pixel values in byte array
            byte[,,] watermark_arr = watermark_img.Data;
            byte[,,] cover_arr = cover.Data;

            //iterate thhrough watermark image
            for (int x = 0; x < watermark_img.Height; x++)
            {
                for (int y = 0; y < watermark_img.Width; y++)
                {
                    //iterate through cover image
                    for (int i = 0; i < cover.Height; i++)
                    {
                        for (int j = 0; j < cover.Width; j++)
                        {
                            int key = (i * cover.Width) + j;
                                if (!blocked.Contains(key)) 
                                {

                                    if ((Math.Abs(watermark_arr[x, y, 0] - cover_arr[i, j, 0]) <= threshold)  //[x,y,0] --> r
                                        && (Math.Abs(watermark_arr[x, y, 1] - cover_arr[i, j, 1]) <= threshold)     //[x,y,1] --> g
                                            && (Math.Abs(watermark_arr[x, y, 2] - cover_arr[i, j, 2]) <= threshold))     //[x,y,2] --> b
                                    {
                                        flag = true;
                                        blocked.Add(key);
                                        count_hit[cnt - 1]++;
                                    }
                            }
                            else
                            {
                                flag = false;
                                count_miss[cnt - 1]++;
                            }
                            button1.Enabled = true;
                            if (flag)
                                break;

                        }
                        if (flag)
                            break;
                    }

                }
            }

        }



    }

    //int min = count_miss.Min();
    //int minIndex = Array.IndexOf(count_miss, min);
    //minIndex++;
    //string currentpath = Application.StartupPath.ToString() + @"\sampleimage\" + minIndex.ToString() + ".png";
    //pictureBox2.Image = Bitmap.FromFile(currentpath);

    if (imageSize)
    {
        int? minVal = null;
        int index = -1;
        for (int i = 0; i < count_miss.Length; i++)
        {
            int thisNum = count_miss[i];
            if (!minVal.HasValue || thisNum < minVal.Value)
            {
                minVal = thisNum;
                index = i;
            }
        }

        //int max = count.Max();
        //int index = count.ToList().IndexOf(max);
        //int index = Array.IndexOf(count, max);

        index = index + 1;
        string path = Application.StartupPath.ToString() + @"\sampleimage\" + index.ToString() + ".png";

        //set pictureBox with selected image
        pictureBox2.Image = Bitmap.FromFile(path);

    }
    else
    {
       // MessageBox.Show("Choose another cover image");
        if (MessageBox.Show("Choose another cover image", "Confirm", MessageBoxButtons.YesNo, MessageBoxIcon.Question) == DialogResult.Yes)
        {
            this.Close();
            new Find_coordinates();
        }
    }
    sp.Stop();
    Debug.WriteLine("Total time (ms): {0}", (long)sp.ElapsedMilliseconds);
}

//embedding -->
private void button1_Click(object sender, EventArgs e)
{

}
 }
 }

Answer 1

So the problem is to match a cover image against several watermark image, and select the watermark image that look mostly similar to the cover image, high levelly speaking.

Code

The method is too long and variable names not very meaningful. E.g. I still have no idea what is a HashSet named blocked. Consider refactoring. But instead of the refactoring the given program, you may want to rewrite it. Because...

Speed

The program tries to loop through all watermark images, for each pixel in the watermark images, compare it with every pixel in the cover image, and count how many pixels are within some threshold in RGB values. Slowness are guaranteed because of its high complexity, and I don't think this does the job. Note that it checks all pixels in the watermark to all pixels in the cover, regardless of which pixel appears to where in the cover image. Even if all pixels in a watermark appears in the cover image doesn't mean it somehow resembles some part of it, because the pixel does not appear in order.

Not double checking a pixel in watermark does not solve the above problem. Also note that the blocked HashSet stored value (i * cover.Width) + j, which is cover image pixel position, not watermark pixel position. It failed in prevent double checking.

Fix

To fix the above code, you should for each watermark image, for each image pixel, align this pixel to the left-top corner of the watermark, and count the pixel values that the difference is under certain threshold.

However this still runs very slowly since the complexity is unchanged, and may not yield satisfactory result. Images are plagued with noise, compression artifacts, camera distortion, zooming in/out etc. Exact matching, unless the images are artificially created, is almost impossible.

Solution

Consider changing the matching criteria. Now this is a very board topic with a whole lot of ways to do it. I would suggest doing a discrete fourier transform, multiple the images in the frequency domain, and pick the coefficient with the highest magnitude. By convolution theorem this is equivalent to computing the correlation between the two images against with every possible alignment. I would also suggest doing it in grayscale since color seldom matters in image comparison.

But I would stop here since this is Code Review not Computer Science. Seek further help there if you want to pursuit deeper.