3
\$\begingroup\$

My goal is to make a class that wraps the complexity of OpenCvSharp implementation to show a webcam streaming into a WPF Image. You can find the complete code with a running example (just clone and compile) on my Github repository.

The important code is this:

public sealed class WebcamStreaming : IDisposable
{
    private System.Drawing.Bitmap _lastFrame;
    private Task _previewTask;

    private CancellationTokenSource _cancellationTokenSource;
    private readonly Image _imageControlForRendering;
    private readonly int _frameWidth;
    private readonly int _frameHeight;

    public int CameraDeviceId { get; private set; }
    public byte[] LastPngFrame { get; private set; }

    public WebcamStreaming(
        Image imageControlForRendering,
        int frameWidth,
        int frameHeight,
        int cameraDeviceId)
    {
        _imageControlForRendering = imageControlForRendering;
        _frameWidth = frameWidth;
        _frameHeight = frameHeight;
        CameraDeviceId = cameraDeviceId;
    }

    public async Task Start()
    {
        // Never run two parallel tasks for the webcam streaming
        if (_previewTask != null && !_previewTask.IsCompleted)
            return;

        var initializationSemaphore = new SemaphoreSlim(0, 1);

        _cancellationTokenSource = new CancellationTokenSource();
        _previewTask = Task.Run(async () =>
        {
            try
            {
                // Creation and disposal of this object should be done in the same thread 
                // because if not it throws disconnectedContext exception
                var videoCapture = new VideoCapture();

                if (!videoCapture.Open(CameraDeviceId))
                {
                    throw new ApplicationException("Cannot connect to camera");
                }

                using (var frame = new Mat())
                {
                    while (!_cancellationTokenSource.IsCancellationRequested)
                    {
                        videoCapture.Read(frame);

                        if (!frame.Empty())
                        {
                            // Releases the lock on first not empty frame
                            if (initializationSemaphore != null)
                                initializationSemaphore.Release();
                            _lastFrame = BitmapConverter.ToBitmap(frame);
                            var lastFrameBitmapImage = _lastFrame.ToBitmapSource();
                            lastFrameBitmapImage.Freeze();
                            _imageControlForRendering.Dispatcher.Invoke(() => _imageControlForRendering.Source = lastFrameBitmapImage);
                        }

                        // 30 FPS
                        await Task.Delay(33);
                    }
                }

                videoCapture?.Dispose();
            }
            finally
            {
                if (initializationSemaphore != null)
                    initializationSemaphore.Release();
            }

        }, _cancellationTokenSource.Token);

        // Async initialization to have the possibility to show an animated loader without freezing the GUI
        // The alternative was the long polling. (while !variable) await Task.Delay
        await initializationSemaphore.WaitAsync();
        initializationSemaphore.Dispose();
        initializationSemaphore = null;

        if (_previewTask.IsFaulted)
        {
            // To let the exceptions exit
            await _previewTask;
        }
    }

    public async Task Stop()
    {
        // If "Dispose" gets called before Stop
        if (_cancellationTokenSource.IsCancellationRequested)
            return;

        if (!_previewTask.IsCompleted)
        {
            _cancellationTokenSource.Cancel();

            // Wait for it, to avoid conflicts with read/write of _lastFrame
            await _previewTask;
        }

        if (_lastFrame != null)
        {
            using (var imageFactory = new ImageFactory())
            using (var stream = new MemoryStream())
            {
                imageFactory
                    .Load(_lastFrame)
                    .Resize(new ResizeLayer(
                        size: new System.Drawing.Size(_frameWidth, _frameHeight),
                        resizeMode: ResizeMode.Crop,
                        anchorPosition: AnchorPosition.Center))
                    .Save(stream);
                LastPngFrame = stream.ToArray();
            }
        }
        else
        {
            LastPngFrame = null;
        }
    }

    public void Dispose()
    {
        _cancellationTokenSource?.Cancel();
        _lastFrame?.Dispose();
    }

}

The usage is this:

  • Create a WPF page, put an Image control in it
  • On a "Start" button, create the WebcamStreaming class and await its initialization:
_webcamStreaming = new WebcamStreaming(
    imageControlForRendering: imageControlReference,
    frameWidth: 300,
    frameHeight: 300,
    cameraDeviceId: cameraDeviceId);

await _webcamStreaming.Start();
  • To stop it, just call: await _webcamStreaming.Stop();

Some considerantions:

  • I used a Semaphore to let the caller await for the first frame, useful to show some loading
  • To render the frame to the WPF Image control I used a synchronous Dispatcher, to avoid frame overlapping, but I'm not sure about that. I think it could cause dealocks, but I never noticed one. Maybe could be better to run & forget an InvokeAsync task?
  • I think I should wait for _previewTask to complete in Dispose method, to avoid disposing something that is in use, but I don't know how to do it without causing deadlocks. Waiting for it would generate deadlock when also waiting for the Dispatcher.Invoke operation to complete.

What do you think? Would you make something different? Thanks!

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy