4
\$\begingroup\$

I am trying to capture the screen content of each monitor I have and divide each captured bitmap into small blocks (wxh=128x128). I managed to do this using the GDI+ API, but my implementation (on Windows using VS2019) is a bit expensive in terms of CPU usage and since I am looping it and constantly capturing the screens, I would like to lower its CPU usage.

So far I managed to use just ~20% on an 8 core System (~2.5% pro core) and if I introduce some sleep in between captures, I can make it go down a bit more but that is not practical.

Is there some further optimizations/ tweaks I can use in my code to lower the CPU usage and have a more efficient code?

#include <windows.h>
#include <vector>
#include <cstring>
#include <string>
#include <iostream>
#include <fstream>

// reference custom/ imported libs headers
#include <Windows.h>
#include <minmax.h>
#include <gdiplus.h>
#include <SDKDDKVer.h>
#pragma comment(lib,"gdiplus.lib")
#include "atlimage.h"

struct cMonitorsVec
{
    std::vector<int>       iMonitors;
    std::vector<HMONITOR>  hMonitors;
    std::vector<HDC>       hdcMonitors;
    std::vector<LPRECT>    lprcMonitors;
    std::vector<RECT>      rcMonitors;

    static BOOL CALLBACK MonitorEnum(HMONITOR hMon, HDC hdc, LPRECT lprcMonitor, LPARAM pData)
    {
        cMonitorsVec* pThis = reinterpret_cast<cMonitorsVec*>(pData);

        pThis->hMonitors.push_back(hMon);
        pThis->hdcMonitors.push_back(hdc);
        pThis->lprcMonitors.push_back(lprcMonitor);
        pThis->rcMonitors.push_back(*lprcMonitor);
        pThis->iMonitors.push_back(pThis->hdcMonitors.size());
        return TRUE;
    }

    cMonitorsVec()
    {
        EnumDisplayMonitors(0, 0, MonitorEnum, (LPARAM)this);
    }
};

bool saveToMemory(HBITMAP* hbitmap, std::vector<BYTE>& data, std::string dataFormat = "png")
{
    Gdiplus::Bitmap bmp(*hbitmap, nullptr);
    // write to IStream
    IStream* istream = nullptr;
    CreateStreamOnHGlobal(NULL, TRUE, &istream);

    // define encoding
    CLSID clsid;
    if (dataFormat.compare("bmp") == 0) { CLSIDFromString(L"{557cf400-1a04-11d3-9a73-0000f81ef32e}", &clsid); }
    else if (dataFormat.compare("jpg") == 0) { CLSIDFromString(L"{557cf401-1a04-11d3-9a73-0000f81ef32e}", &clsid); }
    else if (dataFormat.compare("gif") == 0) { CLSIDFromString(L"{557cf402-1a04-11d3-9a73-0000f81ef32e}", &clsid); }
    else if (dataFormat.compare("tif") == 0) { CLSIDFromString(L"{557cf405-1a04-11d3-9a73-0000f81ef32e}", &clsid); }
    else if (dataFormat.compare("png") == 0) { CLSIDFromString(L"{557cf406-1a04-11d3-9a73-0000f81ef32e}", &clsid); }

    Gdiplus::Status status = bmp.Save(istream, &clsid, NULL);
    if (status != Gdiplus::Status::Ok)
        return false;

    // get memory handle associated with istream
    HGLOBAL hg = NULL;
    GetHGlobalFromStream(istream, &hg);

    // copy IStream to buffer
    int bufsize = GlobalSize(hg);
    data.resize(bufsize);

    // lock & unlock memory
    LPVOID pimage = GlobalLock(hg);
    memcpy(&data[0], pimage, bufsize);
    GlobalUnlock(hg);
    istream->Release();
    return true;
}

BITMAPINFOHEADER createBitmapHeader(int width, int height)
{
    BITMAPINFOHEADER  bi;

    // create a bitmap
    bi.biSize = sizeof(BITMAPINFOHEADER);
    bi.biWidth = width;
    bi.biHeight = -height;  //this is the line that makes it draw upside down or not
    bi.biPlanes = 1;
    bi.biBitCount = 24;
    bi.biCompression = BI_RGB;
    bi.biSizeImage = 0;
    bi.biXPelsPerMeter = 0;
    bi.biYPelsPerMeter = 0;
    bi.biClrUsed = 0;
    bi.biClrImportant = 0;

    return bi;
}

int main()
{
    bool on = TRUE;

    // initilialize GDI+
    CoInitialize(NULL);
    ULONG_PTR token;
    Gdiplus::GdiplusStartupInput tmp;
    Gdiplus::GdiplusStartup(&token, &tmp, NULL);

    int sleepT = 1 * 1000;                                       // init sleep time (ms)
    int capturesCount = 0;

    const int blockWidth = 128;                                 // init block dimensions 
    const int blockHeight = 128;

    std::vector<BYTE> bmpbytes;                                 // init encoded data vectors
    int repetitions = 50;

    for (int r = 0; r < repetitions; r++)
    {
        cMonitorsVec monitorsVec;
        cMonitorsVec* pMonitorsVec = &monitorsVec;
        capturesCount += 1;


        /******************************
         * Capture and post screenshots.
         ******************************/
         // capture screen into blocks, check changes and send block informations
        for (size_t screenshotId = 0; screenshotId < pMonitorsVec->iMonitors.size(); screenshotId++)
        {
            // inits
            HWND chWnd = GetDesktopWindow();
            HWND chwnd = NULL;

            /***************************
             * Capture the whole screen.
             ***************************/
             // get handles to a device context (DC)
            HDC hwindowDC = GetDC(chwnd);
            HDC hwindowCompatibleDC = CreateCompatibleDC(hwindowDC);
            SetStretchBltMode(hwindowCompatibleDC, COLORONCOLOR);

            // init cooredinates, blocks id and block dimensions 
            int y0 = 0;
            int blockId = 0;
            int bwSize, bhSize;
            int screenx = pMonitorsVec->rcMonitors[screenshotId].left;
            int screeny = pMonitorsVec->rcMonitors[screenshotId].top;
            int width = std::abs(pMonitorsVec->rcMonitors[screenshotId].left - pMonitorsVec->rcMonitors[screenshotId].right);
            int height = std::abs(pMonitorsVec->rcMonitors[screenshotId].top - pMonitorsVec->rcMonitors[screenshotId].bottom);

            // std::wcout << "SCREEN WxH=" << std::to_string(width).c_str() << ", " << std::to_string(height).c_str() << std::endl;

            // create a bitmap
            HBITMAP hbwindow = CreateCompatibleBitmap(hwindowDC, width, height);
            BITMAPINFOHEADER bi = createBitmapHeader(width, height);

            // use the previously created device context with the bitmap
            SelectObject(hwindowCompatibleDC, hbwindow);

            // Starting with 32-bit Windows, GlobalAlloc and LocalAlloc are implemented as wrapper functions that call HeapAlloc using a handle to the process's default heap. 
            // Therefore, GlobalAlloc and LocalAlloc have greater overhead than HeapAlloc.
            DWORD dwBmpSize = ((width * bi.biBitCount + 31) / 32) * 4 * height;
            HANDLE hDIB = GlobalAlloc(GHND, dwBmpSize);
            char* lpbitmap = (char*)GlobalLock(hDIB);

            // copy from the window device context to the bitmap device context
            StretchBlt(hwindowCompatibleDC, 0, 0, width, height, hwindowDC, screenx, screeny, width, height, SRCCOPY);   //change SRCCOPY to NOTSRCCOPY for wacky colors !
            GetDIBits(hwindowCompatibleDC, hbwindow, 0, height, lpbitmap, (BITMAPINFO*)&bi, DIB_RGB_COLORS);

            /******************************
            * Divide Screenshot into blocks.
            *******************************/
            while (y0 < height)
            {
                bhSize = ((y0 + blockHeight) > height) * (blockHeight - (y0 + blockHeight - height)) + ((y0 + blockHeight) <= height) * blockHeight;

                int x0 = 0;
                while (x0 < width)
                {
                    bwSize = ((x0 + blockWidth) > width) * (blockWidth - (x0 + blockWidth - width)) + ((x0 + blockWidth) <= width) * blockWidth;

                    HBITMAP tile = CreateCompatibleBitmap(hwindowDC, bwSize, bhSize);              // the bitmap of a single tile
                    HDC tileHdc = CreateCompatibleDC(hwindowDC);                                   // the device that holds the tile bitmap     
                    HBITMAP oldTile = (HBITMAP)SelectObject(tileHdc, tile);                        // insert the tile bitmap into the device context
                    ::BitBlt(tileHdc, 0, 0, bwSize, bhSize, hwindowCompatibleDC, x0, y0, SRCCOPY); // blit from the main bitmap onto the tile bitmap
                    SelectObject(tileHdc, oldTile);                                                // select the tile bitmap

                    std::string dataFormat = "png";
                    // save block
                    if (saveToMemory(&tile, bmpbytes, dataFormat))
                    {
                        std::wcout << "M1: Screenshot saved to memory (X,Y)=" << std::to_string(x0).c_str() << ", " << std::to_string(y0).c_str() << " " << "(W,H)=" << std::to_string(bwSize).c_str() << ", " << std::to_string(bhSize).c_str() << std::endl;

                        // save from memory to file
                        //std::ofstream fout("Screenshots-m1/block_" + std::to_string(y0) + "_" + std::to_string(x0) + "_" + std::to_string(bwSize) + "_" + std::to_string(bhSize) + "." + dataFormat, std::ios::binary);
                        //fout.write((char*)bmpbytes.data(), bmpbytes.size());
                    }
                    else
                        std::wcout << "Error: Couldn't save screenshot to memory" << std::endl;

                    // clean-ups
                    ::SelectObject(tileHdc, tile);
                    ::DeleteDC(tileHdc);
                    ::DeleteObject(tile);
                    ::DeleteObject(oldTile);

                    bmpbytes.clear();
                    blockId += 1;
                    x0 += blockWidth;
                }
                y0 += blockHeight;
            }

            // clean ups
            ::SelectObject(hwindowCompatibleDC, hbwindow); // select the main bitmap which we no longer want
            ::DeleteDC(hwindowCompatibleDC);               // delete the main device context
            ::SelectObject(hwindowDC, hbwindow);           // select the main bitmap which we no longer want
            ::DeleteDC(hwindowDC);                         // delete the main device context
            ::DeleteObject(hbwindow);                      // delete the main bitmap
        }
    }

    // GDI+ clean-ups
    Gdiplus::GdiplusShutdown(token);
    CoUninitialize();
    return 0;
}
\$\endgroup\$

2 Answers 2

3
\$\begingroup\$

So far I managed to use just ~20% on an 8 core System, ( ~2.5% pro core )

That's not how it works, your program is completely single threaded, so it only uses one core. 100% / 4 = ~20% (obviously the maximum is 25%), which is exactly what you're seeing: you're maxing out a core.

First thing I would do is make the image encoding multi-threaded, and absolutely not on the capture thread.

It's not entirely clear what you're trying to achieve with this overall, but if you're saving to memory to display them later and are thinking about GDI+ again, you should probably scrap this and use hardware acceleration (OpenGL, DirectX, Direct2D, etc). And you should reconsider encoding as PNG or whatever you are encoding to if you're just using it as temporary storage, keep the uncompressed bitmap instead.

\$\endgroup\$
3
  • \$\begingroup\$ 100 / 4 = 20 ... hmm \$\endgroup\$
    – user673679
    Commented Jul 28, 2020 at 19:38
  • \$\begingroup\$ Thank you for your comment, but I am not sure how you came up with this 4 nor this 100 / 4 =? 20. As for making the image encoding multi-threaded, can you please elaborate on that? Well I don't need to display the pictures, I am just dividing them into blocks that I later send to a server. The point of dividing the pictures is to only send the changed blocks. As for the PNG encoding, I agree to your point, I am already using Bmp + Zlib or JPEG depending on which variant is smaller. \$\endgroup\$ Commented Jul 29, 2020 at 10:09
  • 1
    \$\begingroup\$ PNG is already zlib encoded, if you're going for storage, then keep the PNG encoding, but absolutely make it multi-threaded. As to elaborating on that, use Task.Run. And as to the 4, what you most likely didn't mention is that you have an 8 core Hyper-Threading CPU, which means you only have 4 physical cores. \$\endgroup\$
    – Blindy
    Commented Jul 29, 2020 at 14:26
1
\$\begingroup\$

Memory Leaks

The application is leaking memory. Visualizing the graphs shows how the memory used constantly goes up, in the hundreds of megabytes.

C++

The code is mostly C. Using C++ property can help with resource management (RAII)

Console Output

Although only used for logging, writing to the console often can have a significant performance cost in this case.

Compressing the image

Saving to PNG consumes most of the CPU time in the case of this application. As others have mentioned, this can be at least parallelized, altough you need to make sure the library used works under multiple threads without issues.

Reusing resources

You create new bitmaps every time. As the screen resolution rarely changes, consider reusing the bitmaps over multiple calls.

Gdiplus::Bitmap bmp(*hbitmap, nullptr); also takes a lot of time. It probably copies the bitmap. This could also be avoided.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.