# OpenCV-based wrapper for Windows screen capture

I have written a ScreenVideoCapture class that implements an interface for capturing frames from a specified monitor connected to the PC. It uses OpenCV structures (including a cv::Mat for image output) and implements the core of the methods that OpenCV's VideoCapture has so that they can be used mostly interchangeably.

The class's constructor/open method take an index that corresponds to the target monitor. It uses the Win32 API to find the proper target rectangle, and then captures from that rectangle when the read method or >> operator are used. Credit goes to this SO question for the basis of the capture code, although it has been modified.

### Specific points of interest:

• Could I structure my class in a way that made it more interchangeable with OpenCV's VideoCapture? Currently, it can be used in the same way as the VideoCapture, but it must be ifdef-ed in in place of the alternative or called separately. There is no way of swapping the implementation at runtime.
• Is there a better way of specifying the target monitor that isn't reliant on the order that EnumDisplayMonitors uses? I remember reading docs that said that the order was constant and determined based on resolution, but I can no longer find that page.
• Can I make the capture more efficient? I am reallocating buffers on each call and do a fair amount of copying from one location to another, but I am not completely sure of what can be done once and what must be done every time.

ScreenVideoCapture.h:

#pragma once

#include <opencv2\opencv.hpp>
#include <Windows.h>

class ScreenVideoCapture
{
public:
ScreenVideoCapture(int displayIndex = -1);
~ScreenVideoCapture();

void open(int displayIndex);
ScreenVideoCapture& operator>>(cv::Mat& destination);

private:
cv::Rect2d captureArea;
HWND targetWindow = NULL;

void captureHwnd(HWND window, cv::Rect2d targetArea, cv::Mat& dest);
static BOOL CALLBACK monitorEnumProc(HMONITOR hMonitor, HDC hdcMonitor, LPRECT lprcMonitor, LPARAM dwData);
};

struct MonitorIndexLookupInfo
{
int targetIndex;

RECT outRect;
int currentIndex;
};


ScreenVideoCapture.cpp:

#include "ScreenVideoCapture.h"

ScreenVideoCapture::ScreenVideoCapture(int displayIndex)
{
if (displayIndex >= 0)
open(displayIndex);
}

void ScreenVideoCapture::open(int displayIndex)
{
MonitorIndexLookupInfo enumState = { displayIndex, NULL, 0 };
EnumDisplayMonitors(NULL, NULL, monitorEnumProc, (LPARAM) &enumState);

this->captureArea = cv::Rect2d(enumState.outRect.left, enumState.outRect.top, enumState.outRect.right - enumState.outRect.left, enumState.outRect.bottom - enumState.outRect.top);
this->targetWindow = GetDesktopWindow();
}

BOOL CALLBACK ScreenVideoCapture::monitorEnumProc(HMONITOR hMonitor, HDC hdcMonitor, LPRECT lprcMonitor, LPARAM dwData)
{
MonitorIndexLookupInfo* enumState = (MonitorIndexLookupInfo*)dwData;
if (enumState->targetIndex == enumState->currentIndex)
{
enumState->outRect = *lprcMonitor;
return false;
}

enumState->currentIndex++;

}

ScreenVideoCapture::~ScreenVideoCapture()
{
}

{
if (targetWindow == NULL)
throw new std::exception("No target monitor specified! The 'open()' method must be called to select a target monitor before frames can be read.");

captureHwnd(targetWindow, captureArea, destination);
}

ScreenVideoCapture& ScreenVideoCapture::operator>>(cv::Mat& destination)
{
return *this;
}

void ScreenVideoCapture::captureHwnd(HWND window, cv::Rect2d targetArea, cv::Mat& dest)
{
HDC hwindowDC, hwindowCompatibleDC;

HBITMAP hbwindow;

hwindowDC = GetDC(window);
hwindowCompatibleDC = CreateCompatibleDC(hwindowDC);
SetStretchBltMode(hwindowCompatibleDC, COLORONCOLOR);

dest.create(targetArea.height, targetArea.width, CV_8UC4);

// Initialize a bitmap
hbwindow = CreateCompatibleBitmap(hwindowDC, targetArea.width, targetArea.height);
bi.biWidth = targetArea.width;
// The negative height is required -- removing the inversion will make the image appear upside-down.
bi.biHeight = -targetArea.height;
bi.biPlanes = 1;
bi.biBitCount = 32;
bi.biCompression = BI_RGB;
bi.biSizeImage = 0;
bi.biXPelsPerMeter = 0;
bi.biYPelsPerMeter = 0;
bi.biClrUsed = 0;
bi.biClrImportant = 0;

SelectObject(hwindowCompatibleDC, hbwindow);
// Copy from the window device context to the bitmap device context
// Use BitBlt to do a copy without any stretching -- the output is of the same dimensions as the target area.
BitBlt(hwindowCompatibleDC, 0, 0, targetArea.width, targetArea.height, hwindowDC, targetArea.x, targetArea.y, SRCCOPY);
// Copy into our own buffer as device-independent bitmap
GetDIBits(hwindowCompatibleDC, hbwindow, 0, targetArea.height, dest.data, (BITMAPINFO *) &bi, DIB_RGB_COLORS);

// Clean up memory to avoid leaks
DeleteObject(hbwindow);
DeleteDC(hwindowCompatibleDC);
ReleaseDC(window, hwindowDC);
}

• How many fps at what resolution were you able to achieve?
– Yoda
Apr 8 '19 at 21:46