タグ: OpenCV

Implementing Background Subtraction (Motion Detection) in WPF with Basler pylon SDK + OpenCV (C# / .NET 8)

Once your live view is working, a natural next step is highlighting only what moved. In this article, we implement a simple background subtraction pipeline to detect motion regions, draw bounding boxes, and display the results in a WPF Image.

✅ Environment

Because this implementation converts between BitmapSource and Mat, install the NuGet package:

OpenCvSharp4.WpfExtensions

Item	Details
Camera	Basler acA2500-14gm
SDK	pylon Camera Software Suite
Language / GUI	C# / .NET 8 / WPF
Libraries	OpenCvSharp4 (`OpenCvSharp4.Windows`, `OpenCvSharp4.WpfExtensions`)

Recommended camera settings (for stable subtraction): Set ExposureAuto=Off, GainAuto=Off, and keep illumination stable (reduce flicker).

Prerequisites from earlier posts (reference):

Implementation Overview

We will follow this workflow:

Capture a background frame (a frame with no motion)
Preprocess with Gaussian blur
Compute absolute difference (AbsDiff) → threshold (Threshold) → opening (MorphologyEx(Open))
Extract contours (FindContours) → draw bounding rectangles
Display the processed frame in WPF

Adding opening after threshold helps remove small speckle noise.

🧩 XAML (Minimal UI)

Add one column for a “Set BG” button and sliders for tuning.

<Grid Grid.Row=”2″ Grid.Column=”0″>
    <Button Content=”Set BG” Width=”80″ Margin=”0,5″ ToolTip=”Capture background”
            Command=”{Binding SetBackgroundCommand}”/>
</Grid>
<Grid Grid.Row=”2″ Grid.Column=”1″ ColumnSpan=”2″ VerticalAlignment=”Center”>
    <StackPanel Orientation=”Horizontal”>
        <TextBlock Text=”Threshold:” Margin=”10,0″/>
        <Slider x:Name=”ThresholdSlider” Minimum=”0″ Maximum=”255″
                Value=”{Binding DetectionThreshold}” Width=”100″ SmallChange=”1″/>
    </StackPanel>
</Grid>
<Grid Grid.Row=”2″ Grid.Column=”3″ ColumnSpan=”2″ VerticalAlignment=”Center”>
    <StackPanel Orientation=”Horizontal”>
        <TextBlock Text=”Min area:” Margin=”10,0″/>
        <Slider x:Name=”MinAreaSlider” Minimum=”200″ Maximum=”100000″
                Value=”{Binding MinArea}” Width=”100″ SmallChange=”100″/>
    </StackPanel>
</Grid>

<Button Content=“Set BG” Width=“80” Margin=“0,5” ToolTip=“Capture background”

Command=“{Binding SetBackgroundCommand}”/>

</Grid>

<Slider x:Name=“ThresholdSlider” Minimum=“0” Maximum=“255”

Value=“{Binding DetectionThreshold}” Width=“100” SmallChange=“1”/>

</StackPanel>

</Grid>

<Slider x:Name=“MinAreaSlider” Minimum=“200” Maximum=“100000”

Value=“{Binding MinArea}” Width=“100” SmallChange=“100”/>

</StackPanel>

</Grid>

BackgroundSubtraction_View

🔧 Core Code (Background Subtraction Pipeline)

This logic ideally belongs in the Model layer, but to keep diffs small from previous articles, it is implemented in the ViewModel.

The author tested on a Mono8 camera. If you use a color camera, convert to grayscale as needed.

The ViewModel implements IDisposable to ensure OpenCV Mat resources are released.

using Basler.Pylon;
using BaslerSamples;
using Common;
using OpenCvSharp;
using OpenCvSharp.WpfExtensions;
using System;
using System.Threading;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Media.Imaging;
using Application = System.Windows.Application;
using Size = OpenCvSharp.Size;

namespace BaslerGUISample.ViewModels
{
    public partial class MainViewModel : BindableBase, IDisposable
    {
        BaslerCameraSample _cameraService = new BaslerCameraSample();

        // Flag: capture the next arriving frame as the background
        private volatile bool _captureBgNext = false;

        public MainViewModel()
        {
            // Other commands omitted
            SetBackgroundCommand = new DelegateCommand(SetBackground, () => IsGrabbing);
        }

        private bool _isGrabbing;
        /// <summary>
        /// Indicates whether the camera is currently grabbing frames.
        /// </summary>
        public bool IsGrabbing
        {
            get => _isGrabbing;
            set => SetProperty(ref _isGrabbing, value);
        }

        private Mat? _backgroundFrame;
        /// <summary>
        /// Background frame used for subtraction.
        /// </summary>
        public Mat? BackgroundFrame
        {
            get => _backgroundFrame;
            set
            {
                var old = _backgroundFrame;
                if (SetProperty(ref _backgroundFrame, value))
                {
                    old?.Dispose();
                    BgBlurCached = null; // Drop blur cache when background changes
                }
            }
        }

        private Mat? _bgBlurCached;
        /// <summary>
        /// Cached blurred background (to avoid blurring every frame).
        /// </summary>
        public Mat? BgBlurCached
        {
            get => _bgBlurCached;
            set
            {
                var old = _bgBlurCached;
                if (SetProperty(ref _bgBlurCached, value))
                {
                    old?.Dispose();
                }
            }
        }

        /// <summary>
        /// True if a background frame has been set.
        /// </summary>
        public bool HasBackground => BackgroundFrame != null;

        private double _detectionThreshold = 30.0;
        /// <summary>
        /// Threshold for motion detection.
        /// </summary>
        public double DetectionThreshold
        {
            get => _detectionThreshold;
            set => SetProperty(ref _detectionThreshold, value);
        }

        private double _minArea = 800.0;
        /// <summary>
        /// Minimum bounding-rect area to keep as a detection.
        /// </summary>
        public double MinArea
        {
            get => _minArea;
            set => SetProperty(ref _minArea, value);
        }

        /// <summary>
        /// Command to capture the background image.
        /// </summary>
        public DelegateCommand SetBackgroundCommand { get; }

        public void Dispose()
        {
            if (IsGrabbing)
            {
                Stop();
            }
            Disconnect();
            BackgroundFrame = null;
            BgBlurCached = null;
        }

        /// <summary>
        /// Capture the next arriving frame as the background.
        /// </summary>
        public void SetBackground()
        {
            _captureBgNext = true;
            BgBlurCached = null;
        }

        private int _updating; // overload protection (drop frames while processing)

        private async void OnImageGrabbed(object? sender, ImageGrabbedEventArgs e)
        {
            IsGrabbing = _cameraService.IsGrabbing;

            if (Interlocked.Exchange(ref _updating, 1) == 1) return;
            try
            {
                await UpdateCurrentFrame(e.GrabResult);
                // HUD update (~10 Hz) omitted
            }
            catch (InvalidOperationException ex)
            {
                MessageBox.Show($”Failed to grab. Error: {ex.Message}”);
            }
            catch (TaskCanceledException)
            {
                // Can occur if the app closes while live view is running.
            }
            finally
            {
                Interlocked.Exchange(ref _updating, 0);
            }
        }

        /// <summary>
        /// Main frame-update routine.
        /// Runs background subtraction, draws bounding boxes, and updates CurrentFrame.
        /// </summary>
        private async Task UpdateCurrentFrame(IGrabResult result)
        {
            if (!result.GrabSucceeded) return;

            // 1) GrabResult -> Mat (reuse implementation from the Mat article)
            using Mat raw = result.ToMat();

            // Ensure single-channel input (safe for color cameras too)
            using Mat frame = new Mat();
            if (raw.Channels() == 1)
            {
                raw.CopyTo(frame);
            }
            else
            {
                Cv2.CvtColor(raw, frame, ColorConversionCodes.BGR2GRAY);
            }

            if (_captureBgNext)
            {
                BackgroundFrame = frame.Clone();
                _captureBgNext = false;
            }

            // If background isn’t set, just display the frame
            if (HasBackground == false)
            {
                await SetCurrentFrame(frame);
                return;
            }

            // 2) Preprocess: blur
            using Mat blur = new();
            Cv2.GaussianBlur(frame, blur, new Size(5, 5), 0);

            // Build blurred background cache if missing
            if (BgBlurCached == null || BgBlurCached.Empty())
            {
                BgBlurCached = new Mat();
                Cv2.GaussianBlur(BackgroundFrame!, BgBlurCached, new Size(5, 5), 0);
            }

            // 3) diff -> threshold -> opening
            using var diff = new Mat();
            Cv2.Absdiff(blur, BgBlurCached, diff);

            using var bw = new Mat();
            Cv2.Threshold(diff, bw, DetectionThreshold, 255, ThresholdTypes.Binary);

            using var k = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(3, 3));
            Cv2.MorphologyEx(bw, bw, MorphTypes.Open, k, iterations: 1);

            // 4) contours -> bounding boxes
            Cv2.FindContours(bw, out var contours, out _, RetrievalModes.External, ContourApproximationModes.ApproxSimple);

            using var colorFrame = new Mat();
            Cv2.CvtColor(frame, colorFrame, ColorConversionCodes.GRAY2BGR);

            int thick = Math.Max(1, colorFrame.Width / 400);
            foreach (var c in contours)
            {
                var rect = Cv2.BoundingRect(c);
                if (rect.Width * rect.Height < MinArea) continue;
                Cv2.Rectangle(colorFrame, rect, Scalar.Red, thick);
            }

            // 5) display
            await SetCurrentFrame(colorFrame);
        }

        private readonly object _lock = new();

        /// <summary>
        /// Converts a Mat to BitmapSource and updates CurrentFrame on the UI thread.
        /// </summary>
        private async Task SetCurrentFrame(Mat mat)
        {
            BitmapSource src = mat.ToBitmapSource();
            src.Freeze(); // Required because it was created on a non-UI thread

            await Application.Current.Dispatcher.BeginInvoke(() =>
            {
                lock (_lock) { CurrentFrame = src; }
            });
        }
    }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

using Basler.Pylon;

using BaslerSamples;

using Common;

using OpenCvSharp;

using OpenCvSharp.WpfExtensions;

using System;

using System.Threading;

using System.Threading.Tasks;

using System.Windows;

using System.Windows.Media.Imaging;

using Application = System.Windows.Application;

using Size = OpenCvSharp.Size;

namespace BaslerGUISample.ViewModels

{

public partial class MainViewModel : BindableBase, IDisposable

{

BaslerCameraSample _cameraService = new BaslerCameraSample();

// Flag: capture the next arriving frame as the background

private volatile bool _captureBgNext = false;

public MainViewModel()

{

// Other commands omitted

SetBackgroundCommand = new DelegateCommand(SetBackground, () => IsGrabbing);

}

private bool _isGrabbing;

/// <summary>

/// Indicates whether the camera is currently grabbing frames.

/// </summary>

public bool IsGrabbing

{

get => _isGrabbing;

set => SetProperty(ref _isGrabbing, value);

}

private Mat? _backgroundFrame;

/// <summary>

/// Background frame used for subtraction.

/// </summary>

public Mat? BackgroundFrame

{

get => _backgroundFrame;

set

{

var old = _backgroundFrame;

if (SetProperty(ref _backgroundFrame, value))

{

old?.Dispose();

BgBlurCached = null; // Drop blur cache when background changes

}

private Mat? _bgBlurCached;

/// <summary>

/// Cached blurred background (to avoid blurring every frame).

/// </summary>

public Mat? BgBlurCached

{

get => _bgBlurCached;

set

{

var old = _bgBlurCached;

if (SetProperty(ref _bgBlurCached, value))

{

old?.Dispose();

}

/// <summary>

/// True if a background frame has been set.

/// </summary>

public bool HasBackground => BackgroundFrame != null;

private double _detectionThreshold = 30.0;

/// <summary>

/// Threshold for motion detection.

/// </summary>

public double DetectionThreshold

{

get => _detectionThreshold;

set => SetProperty(ref _detectionThreshold, value);

}

private double _minArea = 800.0;

/// <summary>

/// Minimum bounding-rect area to keep as a detection.

/// </summary>

public double MinArea

{

get => _minArea;

set => SetProperty(ref _minArea, value);

}

/// <summary>

/// Command to capture the background image.

/// </summary>

public DelegateCommand SetBackgroundCommand { get; }

public void Dispose()

{

if (IsGrabbing)

{

Stop();

}

Disconnect();

BackgroundFrame = null;

BgBlurCached = null;

}

/// <summary>

/// Capture the next arriving frame as the background.

/// </summary>

public void SetBackground()

{

_captureBgNext = true;

BgBlurCached = null;

}

private int _updating; // overload protection (drop frames while processing)

private async void OnImageGrabbed(object? sender, ImageGrabbedEventArgs e)

{

IsGrabbing = _cameraService.IsGrabbing;

if (Interlocked.Exchange(ref _updating, 1) == 1) return;

try

{

await UpdateCurrentFrame(e.GrabResult);

// HUD update (~10 Hz) omitted

}

catch (InvalidOperationException ex)

{

MessageBox.Show($“Failed to grab. Error: {ex.Message}”);

}

catch (TaskCanceledException)

{

// Can occur if the app closes while live view is running.

}

finally

{

Interlocked.Exchange(ref _updating, 0);

}

/// <summary>

/// Main frame-update routine.

/// Runs background subtraction, draws bounding boxes, and updates CurrentFrame.

/// </summary>

private async Task UpdateCurrentFrame(IGrabResult result)

{

if (!result.GrabSucceeded) return;

// 1) GrabResult -> Mat (reuse implementation from the Mat article)

using Mat raw = result.ToMat();

// Ensure single-channel input (safe for color cameras too)

using Mat frame = new Mat();

if (raw.Channels() == 1)

{

raw.CopyTo(frame);

}

else

{

Cv2.CvtColor(raw, frame, ColorConversionCodes.BGR2GRAY);

}

if (_captureBgNext)

{

BackgroundFrame = frame.Clone();

_captureBgNext = false;

}

// If background isn’t set, just display the frame

if (HasBackground == false)

{

await SetCurrentFrame(frame);

return;

}

// 2) Preprocess: blur

using Mat blur = new();

Cv2.GaussianBlur(frame, blur, new Size(5, 5), 0);

// Build blurred background cache if missing

if (BgBlurCached == null || BgBlurCached.Empty())

{

BgBlurCached = new Mat();

Cv2.GaussianBlur(BackgroundFrame!, BgBlurCached, new Size(5, 5), 0);

}

// 3) diff -> threshold -> opening

using var diff = new Mat();

Cv2.Absdiff(blur, BgBlurCached, diff);

using var bw = new Mat();

Cv2.Threshold(diff, bw, DetectionThreshold, 255, ThresholdTypes.Binary);

using var k = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(3, 3));

Cv2.MorphologyEx(bw, bw, MorphTypes.Open, k, iterations: 1);

// 4) contours -> bounding boxes

Cv2.FindContours(bw, out var contours, out _, RetrievalModes.External, ContourApproximationModes.ApproxSimple);

using var colorFrame = new Mat();

Cv2.CvtColor(frame, colorFrame, ColorConversionCodes.GRAY2BGR);

int thick = Math.Max(1, colorFrame.Width / 400);

foreach (var c in contours)

{

var rect = Cv2.BoundingRect(c);

if (rect.Width * rect.Height < MinArea) continue;

Cv2.Rectangle(colorFrame, rect, Scalar.Red, thick);

}

// 5) display

await SetCurrentFrame(colorFrame);

}

private readonly object _lock = new();

/// <summary>

/// Converts a Mat to BitmapSource and updates CurrentFrame on the UI thread.

/// </summary>

private async Task SetCurrentFrame(Mat mat)

{

BitmapSource src = mat.ToBitmapSource();

src.Freeze(); // Required because it was created on a non-UI thread

await Application.Current.Dispatcher.BeginInvoke(() =>

{

lock (_lock) { CurrentFrame = src; }

});

}

Code-Behind

Dispose the ViewModel when the window closes.

/// <summary>
/// Dispose ViewModel before the window closes.
/// </summary>
private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
{
    _viewModel.Dispose();
}

/// <summary>

/// Dispose ViewModel before the window closes.

/// </summary>

private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)

{

_viewModel.Dispose();

}

Example Run

Click Set BG to capture the background frame.

BackgroundSubtraction_BG 2. Move an object in the scene → motion regions are detected and boxed. 3. Use sliders to tune sensitivity and minimum area.

BackgroundSubtraction_Detect

Tuning Tips

Retake background when illumination changes
Threshold: increase to suppress sensor noise; decrease to detect subtle motion
Morphology iterations: increase if edges are jagged; decrease if boxes “bloat”
Min area: simple filter to reduce false positives
Speed-ups:
- Apply camera ROI to reduce the processed area
- Cv2.Resize to process a smaller image (scale coordinates back)
- Cache blurred background (as shown) to avoid blurring every frame

Common Pitfalls

Symptom	Likely Cause	Fix
Nothing detected	Threshold too high / changes too small	Lower threshold or increase blur kernel
Entire frame white	Auto exposure/gain is fluctuating	Set `ExposureAuto/GainAuto=Off`, stabilize lighting
Too many speckles	Sensor noise / tiny vibration	Keep opening (`MorphTypes.Open`) and tune kernel
UI freezes	Processing too heavy	Move processing to another Task, use ROI / resize
Memory grows	`Mat.Dispose()` missing	Use `using` blocks (as shown) and cache carefully

Summary

Background subtraction highlights only moving regions
Fixed exposure/gain + stable lighting improves reliability
Tune threshold, minimum area, and morphology to balance sensitivity vs noise
ROI and resizing are effective for performance

2026-01-19

Displaying and Saving Basler Camera Images with OpenCV

pylon SDK × OpenCvSharp / C#

When working with Basler cameras, converting captured images into OpenCV’s Mat format gives you access to a wide range of image processing algorithms.

In this article, we’ll take a GrabResult from the Basler pylon SDK, convert it to an OpenCV Mat, and then:

Display it in real time using Cv2.ImShow
Save it as an image file using Cv2.ImWrite

✅ Environment

Item	Value
Camera	Basler acA2500-14gm
SDK	Basler pylon Camera Software Suite
Pixel Format	Mono8 (8-bit grayscale)
OpenCV Wrapper	OpenCvSharp4.Windows (via NuGet)

🎯 Goal: Convert `GrabResult` → `Mat` → Display & Save

Conceptual flow:

[GrabResult] → [byte[]] → [Mat] → [ImShow / ImWrite]

🔧 Converting `GrabResult` to `Mat`

We’ll use PixelDataConverter.Convert() to extract pixel data from IGrabResult into a byte[], and then wrap that buffer as an OpenCV Mat.

A convenient way is to implement an extension method on IGrabResult:

// IGrabResultExtensions.cs
using System;
using System.Runtime.InteropServices;
using Basler.Pylon;
using OpenCvSharp;

namespace BaslerSamples
{
    /// <summary>
    /// Extension methods for converting Basler IGrabResult to OpenCV Mat.
    /// </summary>
    public static class IGrabResultExtensions
    {
        /// <summary>
        /// Converts a successful grab result to an OpenCV Mat (Mono8 → CV_8UC1).
        /// </summary>
        /// <param name=”result”>Grabbed image result from the Basler camera.</param>
        /// <returns>Mat instance that wraps the image data.</returns>
        /// <exception cref=”InvalidOperationException”>Thrown if the grab did not succeed.</exception>
        public static Mat ToMat(this IGrabResult result)
        {
            if (!result.GrabSucceeded)
                throw new InvalidOperationException(“Grab failed.”);

            var converter = new PixelDataConverter
            {
                OutputPixelFormat = PixelType.Mono8
            };

            int width = result.Width;
            int height = result.Height;
            byte[] buffer = new byte[width * height];

            converter.Convert(buffer, result);

            // Convert to OpenCV Mat (Gray8 → CV_8UC1)
            GCHandle handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
            try
            {
                IntPtr ptr = handle.AddrOfPinnedObject();
                // Mat.FromPixelData is an OpenCvSharp helper that creates a Mat from raw pixel data.
                return Mat.FromPixelData(height, width, MatType.CV_8UC1, ptr);
            }
            finally
            {
                handle.Free();
            }
        }
    }
}

// IGrabResultExtensions.cs

using System;

using System.Runtime.InteropServices;

using Basler.Pylon;

using OpenCvSharp;

namespace BaslerSamples

{

/// <summary>

/// Extension methods for converting Basler IGrabResult to OpenCV Mat.

/// </summary>

public static class IGrabResultExtensions

{

/// <summary>

/// Converts a successful grab result to an OpenCV Mat (Mono8 → CV_8UC1).

/// </summary>

/// <param name=”result”>Grabbed image result from the Basler camera.</param>

/// <returns>Mat instance that wraps the image data.</returns>

/// <exception cref=”InvalidOperationException”>Thrown if the grab did not succeed.</exception>

public static Mat ToMat(this IGrabResult result)

{

if (!result.GrabSucceeded)

throw new InvalidOperationException(“Grab failed.”);

var converter = new PixelDataConverter

{

OutputPixelFormat = PixelType.Mono8

};

int width = result.Width;

int height = result.Height;

byte[] buffer = new byte[width * height];

converter.Convert(buffer, result);

// Convert to OpenCV Mat (Gray8 → CV_8UC1)

GCHandle handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);

try

{

IntPtr ptr = handle.AddrOfPinnedObject();

// Mat.FromPixelData is an OpenCvSharp helper that creates a Mat from raw pixel data.

return Mat.FromPixelData(height, width, MatType.CV_8UC1, ptr);

}

finally

{

handle.Free();

}

💡 Note: For color formats (e.g., BGR8, RGB8), you’ll need to adjust OutputPixelFormat and MatType (e.g., CV_8UC3) accordingly.

🖼 Real-Time Display with `Cv2.ImShow`

We now extend the BaslerCameraSample class (introduced in the previous article on event-driven capture) to display buffered images using OpenCV.

Here, we assume that _bufferedImageQueue is a queue of (DateTime, IGrabResult) that the ImageGrabbed event handler enqueues into.

using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;
using OpenCvSharp;

public partial class BaslerCameraSample
{
    /// <summary>
    /// Queue for buffering grabbed images for OpenCV display.
    /// </summary>
    private readonly ConcurrentQueue<(DateTime Timestamp, IGrabResult Result)> _bufferedImageQueue
        = new();

    /// <summary>
    /// Displays buffered images using OpenCV’s ImShow until cancellation is requested
    /// and the buffer is drained.
    /// </summary>
    /// <param name=”cts”>Cancellation token source for stopping the display loop.</param>
    public async Task ShowBufferedImages(CancellationTokenSource cts)
    {
        // Loop while capturing is active or there are still items in the buffer.
        while (!cts.IsCancellationRequested && (IsGrabbing || _bufferedImageQueue.Count > 0))
        {
            if (_bufferedImageQueue.TryDequeue(out var item))
            {
                using (Mat mat = item.Result.ToMat())
                {
                    Cv2.ImShow(“Camera”, mat);
                    // WaitKey(0) blocks, so use a short delay (1 ms) for realtime display.
                    Cv2.WaitKey(1);
                }
            }
            else
            {
                // Avoid busy-waiting when the queue is temporarily empty.
                await Task.Delay(1);
            }
        }

        // Optionally clear the queue here.
        _bufferedImageQueue.Clear();
    }
}

using System;

using System.Collections.Concurrent;

using System.Threading;

using System.Threading.Tasks;

using OpenCvSharp;

public partial class BaslerCameraSample

{

/// <summary>

/// Queue for buffering grabbed images for OpenCV display.

/// </summary>

private readonly ConcurrentQueue<(DateTime Timestamp, IGrabResult Result)> _bufferedImageQueue

= new();

/// <summary>

/// Displays buffered images using OpenCV’s ImShow until cancellation is requested

/// and the buffer is drained.

/// </summary>

/// <param name=”cts”>Cancellation token source for stopping the display loop.</param>

public async Task ShowBufferedImages(CancellationTokenSource cts)

{

// Loop while capturing is active or there are still items in the buffer.

while (!cts.IsCancellationRequested && (IsGrabbing || _bufferedImageQueue.Count > 0))

{

if (_bufferedImageQueue.TryDequeue(out var item))

{

using (Mat mat = item.Result.ToMat())

{

Cv2.ImShow(“Camera”, mat);

// WaitKey(0) blocks, so use a short delay (1 ms) for realtime display.

Cv2.WaitKey(1);

}

else

{

// Avoid busy-waiting when the queue is temporarily empty.

await Task.Delay(1);

}

// Optionally clear the queue here.

_bufferedImageQueue.Clear();

}

Example Test: Live Display for 10 Seconds

/// <summary>
/// Tests event-driven grabbing combined with OpenCV realtime display.
/// </summary>
[TestMethod]
public async Task ShowBufferedImagesTest()
{
    if (!_baslerCameraSample.IsConnected)
        _baslerCameraSample.Connect();

    Task? showTask = null;
    var cts = new CancellationTokenSource();

    try
    {
        _baslerCameraSample.StartGrabbing();
        Assert.IsTrue(_baslerCameraSample.IsGrabbing,
            “Camera should be grabbing after StartGrabbing is called.”);

        // Start OpenCV display loop.
        showTask = _baslerCameraSample.ShowBufferedImages(cts);

        // Display camera feed for 10 seconds.
        await Task.Delay(10000);
    }
    catch (InvalidOperationException ex)
    {
        Assert.Fail($”StartGrabbing failed: {ex.Message}”);
    }
    finally
    {
        // Stop grabbing and request display loop to end.
        _baslerCameraSample.StopGrabbing();
        Assert.IsFalse(_baslerCameraSample.IsGrabbing,
            “Camera should not be grabbing after StopGrabbing is called.”);

        cts.Cancel();
        if (showTask is not null)
            await showTask;
    }
}

/// <summary>

/// Tests event-driven grabbing combined with OpenCV realtime display.

/// </summary>

[TestMethod]

public async Task ShowBufferedImagesTest()

{

if (!_baslerCameraSample.IsConnected)

_baslerCameraSample.Connect();

Task? showTask = null;

var cts = new CancellationTokenSource();

try

{

_baslerCameraSample.StartGrabbing();

Assert.IsTrue(_baslerCameraSample.IsGrabbing,

“Camera should be grabbing after StartGrabbing is called.”);

// Start OpenCV display loop.

showTask = _baslerCameraSample.ShowBufferedImages(cts);

// Display camera feed for 10 seconds.

await Task.Delay(10000);

}

catch (InvalidOperationException ex)

{

Assert.Fail($“StartGrabbing failed: {ex.Message}”);

}

finally

{

// Stop grabbing and request display loop to end.

_baslerCameraSample.StopGrabbing();

Assert.IsFalse(_baslerCameraSample.IsGrabbing,

“Camera should not be grabbing after StopGrabbing is called.”);

cts.Cancel();

if (showTask is not null)

await showTask;

}

Running this test opens an OpenCV window named “Camera” that shows the live feed from your Basler camera.

💾 Saving `Mat` with `ImWrite`

To save a Mat to disk, use ImWrite(). The following example:

Captures one frame using Snap()
Converts it to Mat
Saves the image
Applies a simple threshold operation and saves again
Also saves via BitmapSource for comparison

ImShowTest

/// <summary>
/// Captures a single frame, converts it to Mat, saves it, and
/// applies a binary threshold for comparison.
/// </summary>
[TestMethod]
public void SnapAndSaveMatTest()
{
    if (!_baslerCameraSample.IsConnected)
        _baslerCameraSample.Connect();

    // Capture one frame
    using var result = _baslerCameraSample.Snap(1000);
    using (var mat = result.ToMat())
    {
        Assert.IsNotNull(mat);

        // Save the original image using OpenCV
        mat.ImWrite(“SnapAndSaveMatTest_Mat.bmp”);

        // Apply simple thresholding
        Cv2.Threshold(mat, mat, 128, 255, ThresholdTypes.Binary);
        mat.ImWrite(“SnapAndSaveMatTest_Mat_Thresholded.bmp”);

        Assert.IsTrue(System.IO.File.Exists(“SnapAndSaveMatTest_Mat.bmp”));
    }

    // Also save using BitmapSource for comparison
    var bitmap = _baslerCameraSample.ConvertGrabResultToBitmap(result);
    BaslerCameraSample.SaveBitmap(bitmap, “SnapAndSaveMatTest_Bitmap.bmp”);

    Assert.IsNotNull(result);
}

/// <summary>

/// Captures a single frame, converts it to Mat, saves it, and

/// applies a binary threshold for comparison.

/// </summary>

[TestMethod]

public void SnapAndSaveMatTest()

{

if (!_baslerCameraSample.IsConnected)

_baslerCameraSample.Connect();

// Capture one frame

using var result = _baslerCameraSample.Snap(1000);

using (var mat = result.ToMat())

{

Assert.IsNotNull(mat);

// Save the original image using OpenCV

mat.ImWrite(“SnapAndSaveMatTest_Mat.bmp”);

// Apply simple thresholding

Cv2.Threshold(mat, mat, 128, 255, ThresholdTypes.Binary);

mat.ImWrite(“SnapAndSaveMatTest_Mat_Thresholded.bmp”);

Assert.IsTrue(System.IO.File.Exists(“SnapAndSaveMatTest_Mat.bmp”));

}

// Also save using BitmapSource for comparison

var bitmap = _baslerCameraSample.ConvertGrabResultToBitmap(result);

BaslerCameraSample.SaveBitmap(bitmap, “SnapAndSaveMatTest_Bitmap.bmp”);

Assert.IsNotNull(result);

}

This creates three files:

SnapAndSaveMatTest_Bitmap.bmp — saved from BitmapSource
SnapAndSaveMatTest_Mat.bmp — saved from OpenCV Mat
SnapAndSaveMatTest_Mat_Thresholded.bmp — thresholded version

Comparing (1) and (2) confirms that both pipelines produce the same image.

BitmapSource	Mat	Mat(Binary)

📌 Tips & Common Pitfalls

Point	Notes
Pixel format	For color (`BGR8`, `RGB8`), use `MatType.CV_8UC3` and correct channel order
Nothing shows in `ImShow`	You must call `Cv2.WaitKey()`; otherwise the window won’t update
OpenCvSharp packages	Install both `OpenCvSharp4` and `OpenCvSharp4.runtime.windows` via NuGet
Performance considerations	Pinning buffers is fast, but for very high rates you may want to profile allocations

✅ Summary

You can convert Basler pylon GrabResult to an OpenCV Mat via PixelDataConverter and a byte[] buffer.
Cv2.ImShow enables simple live viewing; Cv2.ImWrite lets you save images directly from Mat.
Once in Mat form, you can apply the full power of OpenCV: filtering, edge detection, feature extraction, etc.

Author: @MilleVision

2025-12-10

タグ: OpenCV

Implementing Background Subtraction (Motion Detection) in WPF with Basler pylon SDK + OpenCV (C# / .NET 8)

✅ Environment

Implementation Overview

🧩 XAML (Minimal UI)

🔧 Core Code (Background Subtraction Pipeline)

Code-Behind

Example Run

Tuning Tips

Common Pitfalls

Summary

Displaying and Saving Basler Camera Images with OpenCV

Displaying and Saving Basler Camera Images with OpenCV

✅ Environment

🎯 Goal: Convert GrabResult → Mat → Display & Save

🔧 Converting GrabResult to Mat

🖼 Real-Time Display with Cv2.ImShow

Example Test: Live Display for 10 Seconds

💾 Saving Mat with ImWrite

📌 Tips & Common Pitfalls

✅ Summary

🎯 Goal: Convert `GrabResult` → `Mat` → Display & Save

🔧 Converting `GrabResult` to `Mat`

🖼 Real-Time Display with `Cv2.ImShow`

💾 Saving `Mat` with `ImWrite`