タグ: WPF

Implementing Background Subtraction (Motion Detection) in WPF with Basler pylon SDK + OpenCV (C# / .NET 8)

Once your live view is working, a natural next step is highlighting only what moved. In this article, we implement a simple background subtraction pipeline to detect motion regions, draw bounding boxes, and display the results in a WPF Image.

✅ Environment

Because this implementation converts between BitmapSource and Mat, install the NuGet package:

OpenCvSharp4.WpfExtensions

Item	Details
Camera	Basler acA2500-14gm
SDK	pylon Camera Software Suite
Language / GUI	C# / .NET 8 / WPF
Libraries	OpenCvSharp4 (`OpenCvSharp4.Windows`, `OpenCvSharp4.WpfExtensions`)

Recommended camera settings (for stable subtraction): Set ExposureAuto=Off, GainAuto=Off, and keep illumination stable (reduce flicker).

Prerequisites from earlier posts (reference):

Implementation Overview

We will follow this workflow:

Capture a background frame (a frame with no motion)
Preprocess with Gaussian blur
Compute absolute difference (AbsDiff) → threshold (Threshold) → opening (MorphologyEx(Open))
Extract contours (FindContours) → draw bounding rectangles
Display the processed frame in WPF

Adding opening after threshold helps remove small speckle noise.

🧩 XAML (Minimal UI)

Add one column for a “Set BG” button and sliders for tuning.

<Grid Grid.Row=”2″ Grid.Column=”0″>
    <Button Content=”Set BG” Width=”80″ Margin=”0,5″ ToolTip=”Capture background”
            Command=”{Binding SetBackgroundCommand}”/>
</Grid>
<Grid Grid.Row=”2″ Grid.Column=”1″ ColumnSpan=”2″ VerticalAlignment=”Center”>
    <StackPanel Orientation=”Horizontal”>
        <TextBlock Text=”Threshold:” Margin=”10,0″/>
        <Slider x:Name=”ThresholdSlider” Minimum=”0″ Maximum=”255″
                Value=”{Binding DetectionThreshold}” Width=”100″ SmallChange=”1″/>
    </StackPanel>
</Grid>
<Grid Grid.Row=”2″ Grid.Column=”3″ ColumnSpan=”2″ VerticalAlignment=”Center”>
    <StackPanel Orientation=”Horizontal”>
        <TextBlock Text=”Min area:” Margin=”10,0″/>
        <Slider x:Name=”MinAreaSlider” Minimum=”200″ Maximum=”100000″
                Value=”{Binding MinArea}” Width=”100″ SmallChange=”100″/>
    </StackPanel>
</Grid>

<Button Content=“Set BG” Width=“80” Margin=“0,5” ToolTip=“Capture background”

Command=“{Binding SetBackgroundCommand}”/>

</Grid>

<Slider x:Name=“ThresholdSlider” Minimum=“0” Maximum=“255”

Value=“{Binding DetectionThreshold}” Width=“100” SmallChange=“1”/>

</StackPanel>

</Grid>

<Slider x:Name=“MinAreaSlider” Minimum=“200” Maximum=“100000”

Value=“{Binding MinArea}” Width=“100” SmallChange=“100”/>

</StackPanel>

</Grid>

BackgroundSubtraction_View

🔧 Core Code (Background Subtraction Pipeline)

This logic ideally belongs in the Model layer, but to keep diffs small from previous articles, it is implemented in the ViewModel.

The author tested on a Mono8 camera. If you use a color camera, convert to grayscale as needed.

The ViewModel implements IDisposable to ensure OpenCV Mat resources are released.

using Basler.Pylon;
using BaslerSamples;
using Common;
using OpenCvSharp;
using OpenCvSharp.WpfExtensions;
using System;
using System.Threading;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Media.Imaging;
using Application = System.Windows.Application;
using Size = OpenCvSharp.Size;

namespace BaslerGUISample.ViewModels
{
    public partial class MainViewModel : BindableBase, IDisposable
    {
        BaslerCameraSample _cameraService = new BaslerCameraSample();

        // Flag: capture the next arriving frame as the background
        private volatile bool _captureBgNext = false;

        public MainViewModel()
        {
            // Other commands omitted
            SetBackgroundCommand = new DelegateCommand(SetBackground, () => IsGrabbing);
        }

        private bool _isGrabbing;
        /// <summary>
        /// Indicates whether the camera is currently grabbing frames.
        /// </summary>
        public bool IsGrabbing
        {
            get => _isGrabbing;
            set => SetProperty(ref _isGrabbing, value);
        }

        private Mat? _backgroundFrame;
        /// <summary>
        /// Background frame used for subtraction.
        /// </summary>
        public Mat? BackgroundFrame
        {
            get => _backgroundFrame;
            set
            {
                var old = _backgroundFrame;
                if (SetProperty(ref _backgroundFrame, value))
                {
                    old?.Dispose();
                    BgBlurCached = null; // Drop blur cache when background changes
                }
            }
        }

        private Mat? _bgBlurCached;
        /// <summary>
        /// Cached blurred background (to avoid blurring every frame).
        /// </summary>
        public Mat? BgBlurCached
        {
            get => _bgBlurCached;
            set
            {
                var old = _bgBlurCached;
                if (SetProperty(ref _bgBlurCached, value))
                {
                    old?.Dispose();
                }
            }
        }

        /// <summary>
        /// True if a background frame has been set.
        /// </summary>
        public bool HasBackground => BackgroundFrame != null;

        private double _detectionThreshold = 30.0;
        /// <summary>
        /// Threshold for motion detection.
        /// </summary>
        public double DetectionThreshold
        {
            get => _detectionThreshold;
            set => SetProperty(ref _detectionThreshold, value);
        }

        private double _minArea = 800.0;
        /// <summary>
        /// Minimum bounding-rect area to keep as a detection.
        /// </summary>
        public double MinArea
        {
            get => _minArea;
            set => SetProperty(ref _minArea, value);
        }

        /// <summary>
        /// Command to capture the background image.
        /// </summary>
        public DelegateCommand SetBackgroundCommand { get; }

        public void Dispose()
        {
            if (IsGrabbing)
            {
                Stop();
            }
            Disconnect();
            BackgroundFrame = null;
            BgBlurCached = null;
        }

        /// <summary>
        /// Capture the next arriving frame as the background.
        /// </summary>
        public void SetBackground()
        {
            _captureBgNext = true;
            BgBlurCached = null;
        }

        private int _updating; // overload protection (drop frames while processing)

        private async void OnImageGrabbed(object? sender, ImageGrabbedEventArgs e)
        {
            IsGrabbing = _cameraService.IsGrabbing;

            if (Interlocked.Exchange(ref _updating, 1) == 1) return;
            try
            {
                await UpdateCurrentFrame(e.GrabResult);
                // HUD update (~10 Hz) omitted
            }
            catch (InvalidOperationException ex)
            {
                MessageBox.Show($”Failed to grab. Error: {ex.Message}”);
            }
            catch (TaskCanceledException)
            {
                // Can occur if the app closes while live view is running.
            }
            finally
            {
                Interlocked.Exchange(ref _updating, 0);
            }
        }

        /// <summary>
        /// Main frame-update routine.
        /// Runs background subtraction, draws bounding boxes, and updates CurrentFrame.
        /// </summary>
        private async Task UpdateCurrentFrame(IGrabResult result)
        {
            if (!result.GrabSucceeded) return;

            // 1) GrabResult -> Mat (reuse implementation from the Mat article)
            using Mat raw = result.ToMat();

            // Ensure single-channel input (safe for color cameras too)
            using Mat frame = new Mat();
            if (raw.Channels() == 1)
            {
                raw.CopyTo(frame);
            }
            else
            {
                Cv2.CvtColor(raw, frame, ColorConversionCodes.BGR2GRAY);
            }

            if (_captureBgNext)
            {
                BackgroundFrame = frame.Clone();
                _captureBgNext = false;
            }

            // If background isn’t set, just display the frame
            if (HasBackground == false)
            {
                await SetCurrentFrame(frame);
                return;
            }

            // 2) Preprocess: blur
            using Mat blur = new();
            Cv2.GaussianBlur(frame, blur, new Size(5, 5), 0);

            // Build blurred background cache if missing
            if (BgBlurCached == null || BgBlurCached.Empty())
            {
                BgBlurCached = new Mat();
                Cv2.GaussianBlur(BackgroundFrame!, BgBlurCached, new Size(5, 5), 0);
            }

            // 3) diff -> threshold -> opening
            using var diff = new Mat();
            Cv2.Absdiff(blur, BgBlurCached, diff);

            using var bw = new Mat();
            Cv2.Threshold(diff, bw, DetectionThreshold, 255, ThresholdTypes.Binary);

            using var k = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(3, 3));
            Cv2.MorphologyEx(bw, bw, MorphTypes.Open, k, iterations: 1);

            // 4) contours -> bounding boxes
            Cv2.FindContours(bw, out var contours, out _, RetrievalModes.External, ContourApproximationModes.ApproxSimple);

            using var colorFrame = new Mat();
            Cv2.CvtColor(frame, colorFrame, ColorConversionCodes.GRAY2BGR);

            int thick = Math.Max(1, colorFrame.Width / 400);
            foreach (var c in contours)
            {
                var rect = Cv2.BoundingRect(c);
                if (rect.Width * rect.Height < MinArea) continue;
                Cv2.Rectangle(colorFrame, rect, Scalar.Red, thick);
            }

            // 5) display
            await SetCurrentFrame(colorFrame);
        }

        private readonly object _lock = new();

        /// <summary>
        /// Converts a Mat to BitmapSource and updates CurrentFrame on the UI thread.
        /// </summary>
        private async Task SetCurrentFrame(Mat mat)
        {
            BitmapSource src = mat.ToBitmapSource();
            src.Freeze(); // Required because it was created on a non-UI thread

            await Application.Current.Dispatcher.BeginInvoke(() =>
            {
                lock (_lock) { CurrentFrame = src; }
            });
        }
    }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

using Basler.Pylon;

using BaslerSamples;

using Common;

using OpenCvSharp;

using OpenCvSharp.WpfExtensions;

using System;

using System.Threading;

using System.Threading.Tasks;

using System.Windows;

using System.Windows.Media.Imaging;

using Application = System.Windows.Application;

using Size = OpenCvSharp.Size;

namespace BaslerGUISample.ViewModels

{

public partial class MainViewModel : BindableBase, IDisposable

{

BaslerCameraSample _cameraService = new BaslerCameraSample();

// Flag: capture the next arriving frame as the background

private volatile bool _captureBgNext = false;

public MainViewModel()

{

// Other commands omitted

SetBackgroundCommand = new DelegateCommand(SetBackground, () => IsGrabbing);

}

private bool _isGrabbing;

/// <summary>

/// Indicates whether the camera is currently grabbing frames.

/// </summary>

public bool IsGrabbing

{

get => _isGrabbing;

set => SetProperty(ref _isGrabbing, value);

}

private Mat? _backgroundFrame;

/// <summary>

/// Background frame used for subtraction.

/// </summary>

public Mat? BackgroundFrame

{

get => _backgroundFrame;

set

{

var old = _backgroundFrame;

if (SetProperty(ref _backgroundFrame, value))

{

old?.Dispose();

BgBlurCached = null; // Drop blur cache when background changes

}

private Mat? _bgBlurCached;

/// <summary>

/// Cached blurred background (to avoid blurring every frame).

/// </summary>

public Mat? BgBlurCached

{

get => _bgBlurCached;

set

{

var old = _bgBlurCached;

if (SetProperty(ref _bgBlurCached, value))

{

old?.Dispose();

}

/// <summary>

/// True if a background frame has been set.

/// </summary>

public bool HasBackground => BackgroundFrame != null;

private double _detectionThreshold = 30.0;

/// <summary>

/// Threshold for motion detection.

/// </summary>

public double DetectionThreshold

{

get => _detectionThreshold;

set => SetProperty(ref _detectionThreshold, value);

}

private double _minArea = 800.0;

/// <summary>

/// Minimum bounding-rect area to keep as a detection.

/// </summary>

public double MinArea

{

get => _minArea;

set => SetProperty(ref _minArea, value);

}

/// <summary>

/// Command to capture the background image.

/// </summary>

public DelegateCommand SetBackgroundCommand { get; }

public void Dispose()

{

if (IsGrabbing)

{

Stop();

}

Disconnect();

BackgroundFrame = null;

BgBlurCached = null;

}

/// <summary>

/// Capture the next arriving frame as the background.

/// </summary>

public void SetBackground()

{

_captureBgNext = true;

BgBlurCached = null;

}

private int _updating; // overload protection (drop frames while processing)

private async void OnImageGrabbed(object? sender, ImageGrabbedEventArgs e)

{

IsGrabbing = _cameraService.IsGrabbing;

if (Interlocked.Exchange(ref _updating, 1) == 1) return;

try

{

await UpdateCurrentFrame(e.GrabResult);

// HUD update (~10 Hz) omitted

}

catch (InvalidOperationException ex)

{

MessageBox.Show($“Failed to grab. Error: {ex.Message}”);

}

catch (TaskCanceledException)

{

// Can occur if the app closes while live view is running.

}

finally

{

Interlocked.Exchange(ref _updating, 0);

}

/// <summary>

/// Main frame-update routine.

/// Runs background subtraction, draws bounding boxes, and updates CurrentFrame.

/// </summary>

private async Task UpdateCurrentFrame(IGrabResult result)

{

if (!result.GrabSucceeded) return;

// 1) GrabResult -> Mat (reuse implementation from the Mat article)

using Mat raw = result.ToMat();

// Ensure single-channel input (safe for color cameras too)

using Mat frame = new Mat();

if (raw.Channels() == 1)

{

raw.CopyTo(frame);

}

else

{

Cv2.CvtColor(raw, frame, ColorConversionCodes.BGR2GRAY);

}

if (_captureBgNext)

{

BackgroundFrame = frame.Clone();

_captureBgNext = false;

}

// If background isn’t set, just display the frame

if (HasBackground == false)

{

await SetCurrentFrame(frame);

return;

}

// 2) Preprocess: blur

using Mat blur = new();

Cv2.GaussianBlur(frame, blur, new Size(5, 5), 0);

// Build blurred background cache if missing

if (BgBlurCached == null || BgBlurCached.Empty())

{

BgBlurCached = new Mat();

Cv2.GaussianBlur(BackgroundFrame!, BgBlurCached, new Size(5, 5), 0);

}

// 3) diff -> threshold -> opening

using var diff = new Mat();

Cv2.Absdiff(blur, BgBlurCached, diff);

using var bw = new Mat();

Cv2.Threshold(diff, bw, DetectionThreshold, 255, ThresholdTypes.Binary);

using var k = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(3, 3));

Cv2.MorphologyEx(bw, bw, MorphTypes.Open, k, iterations: 1);

// 4) contours -> bounding boxes

Cv2.FindContours(bw, out var contours, out _, RetrievalModes.External, ContourApproximationModes.ApproxSimple);

using var colorFrame = new Mat();

Cv2.CvtColor(frame, colorFrame, ColorConversionCodes.GRAY2BGR);

int thick = Math.Max(1, colorFrame.Width / 400);

foreach (var c in contours)

{

var rect = Cv2.BoundingRect(c);

if (rect.Width * rect.Height < MinArea) continue;

Cv2.Rectangle(colorFrame, rect, Scalar.Red, thick);

}

// 5) display

await SetCurrentFrame(colorFrame);

}

private readonly object _lock = new();

/// <summary>

/// Converts a Mat to BitmapSource and updates CurrentFrame on the UI thread.

/// </summary>

private async Task SetCurrentFrame(Mat mat)

{

BitmapSource src = mat.ToBitmapSource();

src.Freeze(); // Required because it was created on a non-UI thread

await Application.Current.Dispatcher.BeginInvoke(() =>

{

lock (_lock) { CurrentFrame = src; }

});

}

Code-Behind

Dispose the ViewModel when the window closes.

/// <summary>
/// Dispose ViewModel before the window closes.
/// </summary>
private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
{
    _viewModel.Dispose();
}

/// <summary>

/// Dispose ViewModel before the window closes.

/// </summary>

private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)

{

_viewModel.Dispose();

}

Example Run

Click Set BG to capture the background frame.

BackgroundSubtraction_BG 2. Move an object in the scene → motion regions are detected and boxed. 3. Use sliders to tune sensitivity and minimum area.

BackgroundSubtraction_Detect

Tuning Tips

Retake background when illumination changes
Threshold: increase to suppress sensor noise; decrease to detect subtle motion
Morphology iterations: increase if edges are jagged; decrease if boxes “bloat”
Min area: simple filter to reduce false positives
Speed-ups:
- Apply camera ROI to reduce the processed area
- Cv2.Resize to process a smaller image (scale coordinates back)
- Cache blurred background (as shown) to avoid blurring every frame

Common Pitfalls

Symptom	Likely Cause	Fix
Nothing detected	Threshold too high / changes too small	Lower threshold or increase blur kernel
Entire frame white	Auto exposure/gain is fluctuating	Set `ExposureAuto/GainAuto=Off`, stabilize lighting
Too many speckles	Sensor noise / tiny vibration	Keep opening (`MorphTypes.Open`) and tune kernel
UI freezes	Processing too heavy	Move processing to another Task, use ROI / resize
Memory grows	`Mat.Dispose()` missing	Use `using` blocks (as shown) and cache carefully

Summary

Background subtraction highlights only moving regions
Fixed exposure/gain + stable lighting improves reliability
Tune threshold, minimum area, and morphology to balance sensitivity vs noise
ROI and resizing are effective for performance

2026-01-19

Zoom & Pan for WPF Live View Using TransformGroup (C# / .NET 8)

In the previous article, we built a minimal setup to overlay a HUD (FPS, exposure, gain) on top of live video. As a continuation, this article adds mouse-driven zoom/pan, an ROI rectangle that follows the image, and a crosshair fixed to the screen center.

We keep the same layer-separated design (Video = zoom/pan, HUD = fixed) to minimize blur and jitter.

Environment / Prerequisites

Basler pylon Camera Software Suite (Basler.Pylon referenced)
.NET 8 / WPF (Windows desktop)
You have completed the previous HUD live-view article:
- Live view with CurrentFrame: ImageSource bound to an Image
- HUD overlay

Goals

Mouse wheel zoom centered at the cursor (1.1× per step)
Shift + drag (or middle button drag) to pan
ROI rectangle scales/moves with the image (follows zoom/pan)
Crosshair stays fixed in screen coordinates (HUD layer)

XAML: Two Layers + ROI Layer (Shared Transform)

Because the implementation has grown, some parts are omitted where appropriate.

<Window x:Class=”BaslerGUISample.MainWindow”
        xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
        xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
        xmlns:d=”http://schemas.microsoft.com/expression/blend/2008″
        xmlns:mc=”http://schemas.openxmlformats.org/markup-compatibility/2006″
        xmlns:local=”clr-namespace:BaslerGUISample”
        xmlns:models=”clr-namespace:BaslerGUISample.Models”
        mc:Ignorable=”d”
        Title=”BaslerGUISample” Height=”450″ Width=”500″ Closing=”Window_Closing”>

    <Window.Resources>
        <!– Shared transform for zoom/pan (used by both Image and ROI) –>
        <TransformGroup x:Key=”ContentTransform” Changed=”TransformGroup_Changed”>
            <ScaleTransform x:Name=”Zoom” ScaleX=”1″ ScaleY=”1″/>
            <TranslateTransform x:Name=”Pan” X=”0″ Y=”0″/>
        </TransformGroup>
        <models:HalfConverter x:Key=”Half”/>
    </Window.Resources>
    <Grid>  
        <Grid.RowDefinitions>
            <RowDefinition Height=”Auto”/>
            <RowDefinition Height=”Auto”/>
            <RowDefinition Height=”*”/>
        </Grid.RowDefinitions>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
        </Grid.ColumnDefinitions>

        <!– Row=0 is the same as before –>

        <!– Zoom controls –>
        <Grid Grid.Row=”1″ Grid.Column=”0″>
            <Button Content=”Fit” Width=”80″ Margin=”0,5″ Click=”Fit_Click”/>
        </Grid>
        <Grid Grid.Row=”1″ Grid.Column=”1″>
            <Button Content=”1:1″ Width=”80″ Margin=”0,5″ Click=”Pixel1to1_Click”/>
        </Grid>
        <Grid Grid.Row=”1″ Grid.Column=”2″>
            <Button Content=”Reset” Width=”80″ Margin=”0,5″ Click=”Reset_Click”/>
        </Grid>
        <Grid Grid.Row=”1″ Grid.Column=”3″>
            <TextBlock Text=”{Binding ZoomText, RelativeSource={RelativeSource AncestorType=local:MainWindow}}” VerticalAlignment=”Center” Margin=”8,0,0,0″/>
        </Grid>

        <!– Display area –>
        <Grid Grid.Row=”2″ Grid.Column=”0″ Grid.ColumnSpan=”5″
              ClipToBounds=”True”
              UseLayoutRounding=”True”
              SnapsToDevicePixels=”True”
              TextOptions.TextFormattingMode=”Display”
              TextOptions.TextRenderingMode=”ClearType”
              PreviewMouseWheel=”Root_PreviewMouseWheel”
              MouseLeftButtonDown=”Root_MouseDown”
              MouseMove=”Root_MouseMove”
              MouseUp=”Root_MouseUp”>

            <!– Video layer (zoom/pan applied) –>
            <Canvas x:Name=”ImageLayer”>
                <Image x:Name=”CameraImage”
                       Source=”{Binding CurrentFrame}”
                       RenderOptions.BitmapScalingMode=”NearestNeighbor”
                       Stretch=”Uniform”
                       RenderTransform=”{StaticResource ContentTransform}”/>
            </Canvas>

            <!– ROI layer (shares transform to follow the image) –>
            <Canvas x:Name=”RoiLayer”
                    IsHitTestVisible=”False”
                    RenderTransform=”{StaticResource ContentTransform}”>
                <Rectangle x:Name=”RoiRect” Width=”270″ Height=”200″
                        Canvas.Left=”100″ Canvas.Top=”130″
                        Stroke=”Lime” StrokeThickness=”1″
                        StrokeDashArray=”4,2″ />
            </Canvas>

            <!– HUD layer (screen-fixed, no zoom) –>
            <Grid x:Name=”HudLayer” IsHitTestVisible=”False”>
                <!– Top-left HUD box is the same as previous article –>

                <!– Crosshair fixed at screen center –>
                <Line X1=”0″
                    Y1=”{Binding ActualHeight, ElementName=HudLayer, Converter={StaticResource Half}}”
                    X2=”{Binding ActualWidth,  ElementName=HudLayer}”
                    Y2=”{Binding ActualHeight, ElementName=HudLayer, Converter={StaticResource Half}}”
                    Stroke=”White” StrokeThickness=”1″ StrokeDashArray=”2,2″ Opacity=”0.6″/>
                <Line X1=”{Binding ActualWidth, ElementName=HudLayer, Converter={StaticResource Half}}”
                    Y1=”0″
                    X2=”{Binding ActualWidth, ElementName=HudLayer, Converter={StaticResource Half}}”
                    Y2=”{Binding ActualHeight, ElementName=HudLayer}”
                    Stroke=”White” StrokeThickness=”1″ StrokeDashArray=”2,2″ Opacity=”0.6″/>
                
                <!– Add more HUD elements here –>
            </Grid>
        </Grid>
    </Grid>
</Window>

100

<Window x:Class=“BaslerGUISample.MainWindow”

xmlns=“http://schemas.microsoft.com/winfx/2006/xaml/presentation”

xmlns:x=“http://schemas.microsoft.com/winfx/2006/xaml”

xmlns:d=“http://schemas.microsoft.com/expression/blend/2008”

xmlns:mc=“http://schemas.openxmlformats.org/markup-compatibility/2006”

xmlns:local=“clr-namespace:BaslerGUISample”

xmlns:models=“clr-namespace:BaslerGUISample.Models”

mc:Ignorable=“d”

Title=“BaslerGUISample” Height=“450” Width=“500” Closing=“Window_Closing”>

<Window.Resources>

<!— Shared transform for zoom/pan (used by both Image and ROI) —>

</TransformGroup>

<models:HalfConverter x:Key=“Half”/>

</Window.Resources>

<Grid>

<Grid.RowDefinitions>

</Grid.RowDefinitions>

<Grid.ColumnDefinitions>

</Grid.ColumnDefinitions>

<!— Row=0 is the same as before —>

<!— Zoom controls —>

</Grid>

</Grid>

</Grid>

</Grid>

<!— Display area —>

<Grid Grid.Row=“2” Grid.Column=“0” Grid.ColumnSpan=“5”

ClipToBounds=“True”

UseLayoutRounding=“True”

SnapsToDevicePixels=“True”

TextOptions.TextFormattingMode=“Display”

TextOptions.TextRenderingMode=“ClearType”

PreviewMouseWheel=“Root_PreviewMouseWheel”

MouseLeftButtonDown=“Root_MouseDown”

MouseMove=“Root_MouseMove”

MouseUp=“Root_MouseUp”>

<!— Video layer (zoom/pan applied) —>

<Image x:Name=“CameraImage”

Source=“{Binding CurrentFrame}”

RenderOptions.BitmapScalingMode=“NearestNeighbor”

Stretch=“Uniform”

RenderTransform=“{StaticResource ContentTransform}”/>

</Canvas>

<!— ROI layer (shares transform to follow the image) —>

<Canvas x:Name=“RoiLayer”

IsHitTestVisible=“False”

RenderTransform=“{StaticResource ContentTransform}”>

<Rectangle x:Name=“RoiRect” Width=“270” Height=“200”

Canvas.Left=“100” Canvas.Top=“130”

Stroke=“Lime” StrokeThickness=“1”

StrokeDashArray=“4,2” />

</Canvas>

<!— HUD layer (screen–fixed, no zoom) —>

<!— Top–left HUD box is the same as previous article —>

<!— Crosshair fixed at screen center —>

<Line X1=“0”

Y1=“{Binding ActualHeight, ElementName=HudLayer, Converter={StaticResource Half}}”

X2=“{Binding ActualWidth, ElementName=HudLayer}”

Y2=“{Binding ActualHeight, ElementName=HudLayer, Converter={StaticResource Half}}”

Stroke=“White” StrokeThickness=“1” StrokeDashArray=“2,2” Opacity=“0.6”/>

<Line X1=“{Binding ActualWidth, ElementName=HudLayer, Converter={StaticResource Half}}”

Y1=“0”

X2=“{Binding ActualWidth, ElementName=HudLayer, Converter={StaticResource Half}}”

Y2=“{Binding ActualHeight, ElementName=HudLayer}”

Stroke=“White” StrokeThickness=“1” StrokeDashArray=“2,2” Opacity=“0.6”/>

<!— Add more HUD elements here —>

</Grid>

</Window>

HalfConverter (For Center Crosshair)

This converter makes it easy to draw lines centered in the HUD layer.

using System;
using System.Globalization;
using System.Windows.Data;

namespace BaslerGUISample.Models
{
    public class HalfConverter : IValueConverter
    {
        public object Convert(object value, Type targetType, object parameter, CultureInfo culture)
            => value is double d ? d / 2.0 : 0.0;

        public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)
            => Binding.DoNothing;
    }
}

using System;

using System.Globalization;

using System.Windows.Data;

namespace BaslerGUISample.Models

{

public class HalfConverter : IValueConverter

{

public object Convert(object value, Type targetType, object parameter, CultureInfo culture)

=> value is double d ? d / 2.0 : 0.0;

public object ConvertBack(object value, Type targetType, object parameter, CultureInfo culture)

=> Binding.DoNothing;

}

Code-Behind: Zoom / Pan

In this article, zoom and pan are implemented purely in code-behind. You can refactor into ViewModel/Model later if needed.

using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using BaslerGUISample.ViewModels;

namespace BaslerGUISample
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        private readonly MainViewModel _viewModel;

        private TransformGroup? _contentTransform;
        private ScaleTransform? _scale;
        private TranslateTransform? _trans;

        private bool _panning;
        private Point _panStart;

        public MainWindow()
        {
            InitializeComponent();
            _viewModel = new MainViewModel();
            DataContext = _viewModel;
            _contentTransform = (TransformGroup)Resources[“ContentTransform”];
            _scale = (ScaleTransform)_contentTransform.Children[0];
            _trans = (TranslateTransform)_contentTransform.Children[1];
            UpdateZoomText();
        }

        public string ZoomText
        {
            get => (string)GetValue(ZoomTextProperty);
            set => SetValue(ZoomTextProperty, value);
        }
        public static readonly DependencyProperty ZoomTextProperty =
            DependencyProperty.Register(nameof(ZoomText), typeof(string), typeof(MainWindow),
                new PropertyMetadata(“Zoom: 1.00x”));

        /// <summary>
        /// Disconnect the camera before the window closes.
        /// </summary>
        private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
        {
            if (_viewModel.IsGrabbing)
            {
                _viewModel.Stop();
            }
            _viewModel.Disconnect();
        }
        
        private void Root_PreviewMouseWheel(object sender, MouseWheelEventArgs e)
        {
            const double step = 1.1;
            double factor = e.Delta > 0 ? step : 1.0 / step;

            var viewPt    = e.GetPosition(ImageLayer);
            var contentPt = ToContent(viewPt);

            double newScaleX = _scale!.ScaleX * factor;
            double newScaleY = _scale!.ScaleY * factor;
            newScaleX = Math.Clamp(newScaleX, 0.1, 20.0);
            newScaleY = Math.Clamp(newScaleY, 0.1, 20.0);
            _scale.ScaleX = newScaleX;
            _scale.ScaleY = newScaleY;

            var post = _contentTransform!.Transform(contentPt);
            _trans!.X += viewPt.X – post.X;
            _trans!.Y += viewPt.Y – post.Y;

        }

        private void Root_MouseDown(object sender, MouseButtonEventArgs e)
        {
            if (e.MiddleButton == MouseButtonState.Pressed || (e.LeftButton == MouseButtonState.Pressed && (Keyboard.IsKeyDown(Key.LeftShift) || Keyboard.IsKeyDown(Key.RightShift))))
            {
                _panning = true;
                _panStart = e.GetPosition(ImageLayer);
                Mouse.Capture(ImageLayer);
            }
        }

        private void Root_MouseMove(object sender, MouseEventArgs e)
        {
            if (!_panning) return;
            var now = e.GetPosition(ImageLayer);
            var delta = now – _panStart;
            _panStart = now;
            _trans!.X += delta.X;
             _trans!.Y += delta.Y;
        }

        private void Root_MouseUp(object sender, MouseButtonEventArgs e)
        {
            _panning = false;
            Mouse.Capture(null);
        }

        private void Reset_Click(object sender, RoutedEventArgs e)
        {
            _scale!.ScaleX = _scale.ScaleY = 1.0;
            _trans!.X = _trans.Y = 0.0;
        }

        private void Pixel1to1_Click(object sender, RoutedEventArgs e)
        {
            _scale!.ScaleX = _scale.ScaleY = 1.0;
        }

        private void Fit_Click(object sender, RoutedEventArgs e)
        {
            if (_viewModel.CurrentFrame is null)
                return;
            var w = ImageLayer.ActualWidth;
            var h = ImageLayer.ActualHeight;
            if (w <= 0 || h <= 0) return;

            double srcW = _viewModel.CurrentFrame.PixelWidth, srcH = _viewModel.CurrentFrame.PixelHeight;
            double s = Math.Min(w / srcW, h / srcH);
            _scale!.ScaleX = _scale.ScaleY = s;
            _trans!.X = (w – srcW * s) * 0.5;
            _trans!.Y = (h – srcH * s) * 0.5;
        }

        private void UpdateZoomText()
        {
            ZoomText = $”Zoom: {_scale?.ScaleX:0.00}x”;
        }

        private Point ToContent(Point viewPt)
        {
            var inv = _contentTransform!.Inverse;
            return inv.Transform(viewPt);
        }

        private void TransformGroup_Changed(object sender, EventArgs e)
        {
            UpdateZoomText();
        }
    }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

using System.Windows;

using System.Windows.Input;

using System.Windows.Media;

using BaslerGUISample.ViewModels;

namespace BaslerGUISample

{

/// <summary>

/// Interaction logic for MainWindow.xaml

/// </summary>

public partial class MainWindow : Window

{

private readonly MainViewModel _viewModel;

private TransformGroup? _contentTransform;

private ScaleTransform? _scale;

private TranslateTransform? _trans;

private bool _panning;

private Point _panStart;

public MainWindow()

{

InitializeComponent();

_viewModel = new MainViewModel();

DataContext = _viewModel;

_contentTransform = (TransformGroup)Resources[“ContentTransform”];

_scale = (ScaleTransform)_contentTransform.Children[0];

_trans = (TranslateTransform)_contentTransform.Children[1];

UpdateZoomText();

}

public string ZoomText

{

get => (string)GetValue(ZoomTextProperty);

set => SetValue(ZoomTextProperty, value);

}

public static readonly DependencyProperty ZoomTextProperty =

DependencyProperty.Register(nameof(ZoomText), typeof(string), typeof(MainWindow),

new PropertyMetadata(“Zoom: 1.00x”));

/// <summary>

/// Disconnect the camera before the window closes.

/// </summary>

private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)

{

if (_viewModel.IsGrabbing)

{

_viewModel.Stop();

}

_viewModel.Disconnect();

}

private void Root_PreviewMouseWheel(object sender, MouseWheelEventArgs e)

{

const double step = 1.1;

double factor = e.Delta > 0 ? step : 1.0 / step;

var viewPt = e.GetPosition(ImageLayer);

var contentPt = ToContent(viewPt);

double newScaleX = _scale!.ScaleX * factor;

double newScaleY = _scale!.ScaleY * factor;

newScaleX = Math.Clamp(newScaleX, 0.1, 20.0);

newScaleY = Math.Clamp(newScaleY, 0.1, 20.0);

_scale.ScaleX = newScaleX;

_scale.ScaleY = newScaleY;

var post = _contentTransform!.Transform(contentPt);

_trans!.X += viewPt.X – post.X;

_trans!.Y += viewPt.Y – post.Y;

}

private void Root_MouseDown(object sender, MouseButtonEventArgs e)

{

if (e.MiddleButton == MouseButtonState.Pressed || (e.LeftButton == MouseButtonState.Pressed && (Keyboard.IsKeyDown(Key.LeftShift) || Keyboard.IsKeyDown(Key.RightShift))))

{

_panning = true;

_panStart = e.GetPosition(ImageLayer);

Mouse.Capture(ImageLayer);

}

private void Root_MouseMove(object sender, MouseEventArgs e)

{

if (!_panning) return;

var now = e.GetPosition(ImageLayer);

var delta = now – _panStart;

_panStart = now;

_trans!.X += delta.X;

_trans!.Y += delta.Y;

}

private void Root_MouseUp(object sender, MouseButtonEventArgs e)

{

_panning = false;

Mouse.Capture(null);

}

private void Reset_Click(object sender, RoutedEventArgs e)

{

_scale!.ScaleX = _scale.ScaleY = 1.0;

_trans!.X = _trans.Y = 0.0;

}

private void Pixel1to1_Click(object sender, RoutedEventArgs e)

{

_scale!.ScaleX = _scale.ScaleY = 1.0;

}

private void Fit_Click(object sender, RoutedEventArgs e)

{

if (_viewModel.CurrentFrame is null)

return;

var w = ImageLayer.ActualWidth;

var h = ImageLayer.ActualHeight;

if (w <= 0 || h <= 0) return;

double srcW = _viewModel.CurrentFrame.PixelWidth, srcH = _viewModel.CurrentFrame.PixelHeight;

double s = Math.Min(w / srcW, h / srcH);

_scale!.ScaleX = _scale.ScaleY = s;

_trans!.X = (w – srcW * s) * 0.5;

_trans!.Y = (h – srcH * s) * 0.5;

}

private void UpdateZoomText()

{

ZoomText = $“Zoom: {_scale?.ScaleX:0.00}x”;

}

private Point ToContent(Point viewPt)

{

var inv = _contentTransform!.Inverse;

return inv.Transform(viewPt);

}

private void TransformGroup_Changed(object sender, EventArgs e)

{

UpdateZoomText();

}

ROI Handling and Applying It in Real Apps

In this sample, we assign the same TransformGroup to both the Image and the RoiLayer.

Because RoiLayer shares the transform, it naturally follows zoom/pan.
In real applications, you typically bind Canvas.Left/Top/Width/Height of RoiRect to a ViewModel, then apply those values to camera ROI parameters.
When applying ROI to the camera, remember GenICam constraints such as width/height increments.

Meanwhile, HUD elements (crosshair, texts) remain transform-free and screen-fixed, preventing blur and keeping interaction intuitive.

Example Screens

The UI looks like this:

Pan the live view by dragging while holding Shift. The ROI follows; the crosshair stays fixed.

Zoom with the mouse wheel. The zoom ratio in the upper area updates as well.

Use Fit to fit the image to the window. 1:1 sets the zoom to 1.0, and Reset returns zoom to 1.0 and pan offset to 0.

Summary

Video + ROI share the same transform; HUD stays fixed (layered design)
Mouse interaction provides cursor-centered zoom and intuitive pan
ROI follows the image while the crosshair remains screen-fixed

2025-12-26

Overlaying a HUD on Basler Live View in WPF (FPS, Exposure, Gain) — Dual-Layer UI Layout

Once you have a working live preview, the next step is showing camera state in a readable way—especially when screen space is limited (embedded displays, lab setups, compact inspection tools).

In this article, we overlay a HUD (FPS, exposure, gain, crosshair, etc.) on top of a Basler live view in WPF. Instead of drawing text directly onto the image (which can blur or jitter during zoom/pan), we use a layer-separated approach: an image layer for the video and a transparent HUD layer that stays fixed in screen coordinates.

This structure is also a solid foundation for future features such as:

ROI (Region of Interest) editing in the GUI
Capturing and saving “image + HUD” using RenderTargetBitmap

Environment / Assumptions

Basler pylon Camera Software Suite (Basler.Pylon referenced)
.NET 8 / WPF (Windows desktop)
Camera: acA2500-14gm (assume Mono8)

Goal

Building on the previous live-view article, we add:

A two-layer layout: Video layer (Image) + HUD layer (transparent Grid)
The HUD is fixed to screen coordinates (does not follow zoom), so text does not blur/jitter
FPS calculated continuously but UI-updated at ~10 Hz, plus exposure/gain display
A design that can later evolve into HUD-included capture via RenderTargetBitmap

XAML: Dual Layer (with Optional ROI Layer)

Inside the display Grid, we stack multiple layers:

Video layer (can be zoomed/panned in future)
ROI layer (optional, follows the image)
HUD layer (screen-fixed overlay)

<Window x:Class=”BaslerGUISample.MainWindow”
        xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
        xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
        xmlns:d=”http://schemas.microsoft.com/expression/blend/2008″
        xmlns:mc=”http://schemas.openxmlformats.org/markup-compatibility/2006″
        xmlns:local=”clr-namespace:BaslerGUISample”
        mc:Ignorable=”d”
        Title=”BaslerGUISample” Height=”450″ Width=”500″ Closing=”Window_Closing”>

    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition Height=”Auto”/>
            <RowDefinition Height=”*”/>
        </Grid.RowDefinitions>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
        </Grid.ColumnDefinitions>
        <!– Buttons omitted –>

        <Grid Grid.Row=”1″ Grid.Column=”0″ Grid.ColumnSpan=”5″
              UseLayoutRounding=”True”
              SnapsToDevicePixels=”True”
              TextOptions.TextFormattingMode=”Display”
              TextOptions.TextRenderingMode=”ClearType”>

            <!– Video layer (zoom/pan applied here in future) –>
            <Canvas x:Name=”ImageLayer”>
                <Image x:Name=”CameraImage”
                       Source=”{Binding CurrentFrame}”
                       RenderOptions.BitmapScalingMode=”NearestNeighbor”
                       Stretch=”Uniform”/>
            </Canvas>

            <!– ROI layer (optional; follows the image) –>
            <Canvas x:Name=”RoiLayer” IsHitTestVisible=”False”>
            </Canvas>

            <!– HUD layer (screen-fixed; does not follow zoom) –>
            <Grid x:Name=”HudLayer” IsHitTestVisible=”False”>
                <Border Background=”#66000000″ CornerRadius=”6″ Padding=”8″ Margin=”8″
                        HorizontalAlignment=”Left” VerticalAlignment=”Top”>
                    <StackPanel>
                        <TextBlock Text=”{Binding FpsText}”      Foreground=”White” FontSize=”14″/>
                        <TextBlock Text=”{Binding ExposureText}” Foreground=”White” FontSize=”12″/>
                        <TextBlock Text=”{Binding GainText}”     Foreground=”White” FontSize=”12″/>
                    </StackPanel>
                </Border>

                <!– Add more HUD elements here –>
            </Grid>
        </Grid>
    </Grid>
</Window>

<Window x:Class=“BaslerGUISample.MainWindow”

xmlns=“http://schemas.microsoft.com/winfx/2006/xaml/presentation”

xmlns:x=“http://schemas.microsoft.com/winfx/2006/xaml”

xmlns:d=“http://schemas.microsoft.com/expression/blend/2008”

xmlns:mc=“http://schemas.openxmlformats.org/markup-compatibility/2006”

xmlns:local=“clr-namespace:BaslerGUISample”

mc:Ignorable=“d”

Title=“BaslerGUISample” Height=“450” Width=“500” Closing=“Window_Closing”>

<Grid>

<Grid.RowDefinitions>

</Grid.RowDefinitions>

<Grid.ColumnDefinitions>

</Grid.ColumnDefinitions>

<!— Buttons omitted —>

<Grid Grid.Row=“1” Grid.Column=“0” Grid.ColumnSpan=“5”

UseLayoutRounding=“True”

SnapsToDevicePixels=“True”

TextOptions.TextFormattingMode=“Display”

TextOptions.TextRenderingMode=“ClearType”>

<!— Video layer (zoom/pan applied here in future) —>

<Image x:Name=“CameraImage”

Source=“{Binding CurrentFrame}”

RenderOptions.BitmapScalingMode=“NearestNeighbor”

Stretch=“Uniform”/>

</Canvas>

<!— ROI layer (optional; follows the image) —>

</Canvas>

<!— HUD layer (screen–fixed; does not follow zoom) —>

<Border Background=“#66000000” CornerRadius=“6” Padding=“8” Margin=“8”

HorizontalAlignment=“Left” VerticalAlignment=“Top”>

</StackPanel>

</Border>

<!— Add more HUD elements here —>

</Grid>

</Window>

NearestNeighbor is good when you want pixel-accurate rendering.
Set IsHitTestVisible="False" so the HUD doesn’t steal mouse input (important for future zoom/pan).

ViewModel: Throttled FPS Updates + Exposure/Gain HUD

We extend the previous live-view ViewModel with:

FPS measurement using a Stopwatch
HUD updates throttled to ~10 Hz
Exposure and gain values displayed alongside FPS

Connect / Disconnect are omitted here—refer to your earlier WPF snapshot article.

using Basler.Pylon;
using BaslerSamples;
using Common;
using System;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Media;
using Application = System.Windows.Application;

namespace BaslerGUISample.ViewModels
{
    public class MainViewModel : BindableBase
    {
        BaslerCameraSample _cameraService = new BaslerCameraSample();

        // Fields for HUD calculation
        private readonly Stopwatch _fpsSw = new();
        private int _frameCount;
        private long _lastHudUpdateMs;

        private bool _isConnected;
        public bool IsConnected
        {
            get => _isConnected;
            set { SetProperty(ref _isConnected, value); }
        }

        private bool _isGrabbing;
        public bool IsGrabbing
        {
            get => _isGrabbing;
            set => SetProperty(ref _isGrabbing, value);
        }

        private ImageSource? _currentFrame;
        public ImageSource? CurrentFrame
        {
            get => _currentFrame;
            set => SetProperty(ref _currentFrame, value);
        }

        // HUD text
        public string FpsText
        {
            get => _fpsText;
            private set => SetProperty(ref _fpsText, value);
        }
        public string ExposureText
        {
            get => _expText;
            private set => SetProperty(ref _expText, value);
        }
        public string GainText
        {
            get => _gainText;
            private set => SetProperty(ref _gainText, value);
        }

        private string _fpsText = “FPS: –.-“;
        private string _expText = “Exposure: — μs”;
        private string _gainText = “Gain: — dB”;

        public DelegateCommand StartCommand { get; }
        public DelegateCommand StopCommand  { get; }

        public MainViewModel()
        {
            StartCommand = new DelegateCommand(Start, () => IsConnected && !IsGrabbing);
            StopCommand  = new DelegateCommand(Stop,  () => IsConnected && IsGrabbing);
        }

        public void Start()
        {
            if (IsConnected == false) return;

            // Subscribe (remove once to avoid duplicates)
            _cameraService.RemoveImageGrabbedEventHandler(OnImageGrabbed);
            _cameraService.AddImageGrabbedEventHandler(OnImageGrabbed);

            _cameraService.StartLive();
            IsGrabbing = _cameraService.IsGrabbing;

            // HUD init
            _fpsSw.Restart();
            _lastHudUpdateMs = 0;
            Interlocked.Exchange(ref _frameCount, 0);
        }

        public void Stop()
        {
            if (_cameraService.IsConnected == false) return;

            _cameraService.StopGrabbing();
            _cameraService.RemoveImageGrabbedEventHandler(OnImageGrabbed);
            IsGrabbing = _cameraService.IsGrabbing;
        }

        private int _updating; // overload protection

        /// <summary>
        /// Called when a frame is received.
        /// Updates the video frame every time, and updates HUD at ~10 Hz.
        /// </summary>
        private async void OnImageGrabbed(object? sender, ImageGrabbedEventArgs e)
        {
            IsGrabbing = _cameraService.IsGrabbing;

            // Overload protection: drop frames while processing UI update
            if (Interlocked.Exchange(ref _updating, 1) == 1) return;

            try
            {
                using IGrabResult result = e.GrabResult.Clone();

                // Video update (switch to UI thread)
                await Application.Current.Dispatcher.InvokeAsync(() =>
                {
                    var frame = BaslerCameraSample.ConvertGrabResultToBitmap(result);
                    CurrentFrame = frame;
                });

                // Count frames
                Interlocked.Increment(ref _frameCount);

                // HUD update (~10 Hz)
                var nowMs = _fpsSw.ElapsedMilliseconds;
                if (nowMs – _lastHudUpdateMs >= 100)
                {
                    int count = Interlocked.Exchange(ref _frameCount, 0);
                    double fps = count * 1000.0 / Math.Max(1, nowMs – _lastHudUpdateMs);
                    _lastHudUpdateMs = nowMs;

                    // Read exposure/gain (catch differences between models)
                    string expText = _expText, gainText = _gainText;

                    try
                    {
                        double exp = _cameraService.GetExposureTime();
                        expText = double.IsNaN(exp) ? “Exposure: — μs” : $”Exposure: {exp:0} μs”;
                    }
                    catch { /* keep previous */ }

                    try
                    {
                        // If Gain is not supported, use GainRaw instead
                        double gain = _cameraService.GetGainRaw();
                        gainText = double.IsNaN(gain) ? “Gain: –” : $”Gain: {gain:0.0}”;
                    }
                    catch { /* keep previous */ }

                    Application.Current?.Dispatcher.Invoke(() =>
                    {
                        FpsText = $”FPS: {fps:0.0}”;
                        ExposureText = expText;
                        GainText = gainText;
                    });
                }
            }
            catch (InvalidOperationException ex)
            {
                MessageBox.Show($”Failed to grab. Error: {ex.Message}”);
            }
            catch (TaskCanceledException)
            {
                // If the app closes during live view, this can occur.
            }
            finally
            {
                Interlocked.Exchange(ref _updating, 0);
            }
        }
    }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

using Basler.Pylon;

using BaslerSamples;

using Common;

using System;

using System.Diagnostics;

using System.Threading;

using System.Threading.Tasks;

using System.Windows;

using System.Windows.Media;

using Application = System.Windows.Application;

namespace BaslerGUISample.ViewModels

{

public class MainViewModel : BindableBase

{

BaslerCameraSample _cameraService = new BaslerCameraSample();

// Fields for HUD calculation

private readonly Stopwatch _fpsSw = new();

private int _frameCount;

private long _lastHudUpdateMs;

private bool _isConnected;

public bool IsConnected

{

get => _isConnected;

set { SetProperty(ref _isConnected, value); }

}

private bool _isGrabbing;

public bool IsGrabbing

{

get => _isGrabbing;

set => SetProperty(ref _isGrabbing, value);

}

private ImageSource? _currentFrame;

public ImageSource? CurrentFrame

{

get => _currentFrame;

set => SetProperty(ref _currentFrame, value);

}

// HUD text

public string FpsText

{

get => _fpsText;

private set => SetProperty(ref _fpsText, value);

}

public string ExposureText

{

get => _expText;

private set => SetProperty(ref _expText, value);

}

public string GainText

{

get => _gainText;

private set => SetProperty(ref _gainText, value);

}

private string _fpsText = “FPS: –.-“;

private string _expText = “Exposure: — μs”;

private string _gainText = “Gain: — dB”;

public DelegateCommand StartCommand { get; }

public DelegateCommand StopCommand { get; }

public MainViewModel()

{

StartCommand = new DelegateCommand(Start, () => IsConnected && !IsGrabbing);

StopCommand = new DelegateCommand(Stop, () => IsConnected && IsGrabbing);

}

public void Start()

{

if (IsConnected == false) return;

// Subscribe (remove once to avoid duplicates)

_cameraService.RemoveImageGrabbedEventHandler(OnImageGrabbed);

_cameraService.AddImageGrabbedEventHandler(OnImageGrabbed);

_cameraService.StartLive();

IsGrabbing = _cameraService.IsGrabbing;

// HUD init

_fpsSw.Restart();

_lastHudUpdateMs = 0;

Interlocked.Exchange(ref _frameCount, 0);

}

public void Stop()

{

if (_cameraService.IsConnected == false) return;

_cameraService.StopGrabbing();

_cameraService.RemoveImageGrabbedEventHandler(OnImageGrabbed);

IsGrabbing = _cameraService.IsGrabbing;

}

private int _updating; // overload protection

/// <summary>

/// Called when a frame is received.

/// Updates the video frame every time, and updates HUD at ~10 Hz.

/// </summary>

private async void OnImageGrabbed(object? sender, ImageGrabbedEventArgs e)

{

IsGrabbing = _cameraService.IsGrabbing;

// Overload protection: drop frames while processing UI update

if (Interlocked.Exchange(ref _updating, 1) == 1) return;

try

{

using IGrabResult result = e.GrabResult.Clone();

// Video update (switch to UI thread)

await Application.Current.Dispatcher.InvokeAsync(() =>

{

var frame = BaslerCameraSample.ConvertGrabResultToBitmap(result);

CurrentFrame = frame;

});

// Count frames

Interlocked.Increment(ref _frameCount);

// HUD update (~10 Hz)

var nowMs = _fpsSw.ElapsedMilliseconds;

if (nowMs – _lastHudUpdateMs >= 100)

{

int count = Interlocked.Exchange(ref _frameCount, 0);

double fps = count * 1000.0 / Math.Max(1, nowMs – _lastHudUpdateMs);

_lastHudUpdateMs = nowMs;

// Read exposure/gain (catch differences between models)

string expText = _expText, gainText = _gainText;

try

{

double exp = _cameraService.GetExposureTime();

expText = double.IsNaN(exp) ? “Exposure: — μs” : $“Exposure: {exp:0} μs”;

}

catch { /* keep previous */ }

try

{

// If Gain is not supported, use GainRaw instead

double gain = _cameraService.GetGainRaw();

gainText = double.IsNaN(gain) ? “Gain: –“ : $“Gain: {gain:0.0}”;

}

catch { /* keep previous */ }

Application.Current?.Dispatcher.Invoke(() =>

{

FpsText = $“FPS: {fps:0.0}”;

ExposureText = expText;

GainText = gainText;

});

}

catch (InvalidOperationException ex)

{

MessageBox.Show($“Failed to grab. Error: {ex.Message}”);

}

catch (TaskCanceledException)

{

// If the app closes during live view, this can occur.

}

finally

{

Interlocked.Exchange(ref _updating, 0);

}

Example Output

You’ll see FPS, exposure, and gain values displayed in the upper-left corner of the live preview. Live view makes FPS variations surprisingly fun to watch—feel free to customize your HUD further (movable HUD, crosshair, etc.).

HUD sample

Common Pitfalls & Fixes

Text looks blurry/jitters → Keep HUD on a separate layer (screen-fixed). Enable UseLayoutRounding, SnapsToDevicePixels, and ClearType settings.
UI is sluggish → Update video quickly; throttle HUD updates to ~10 Hz. Do heavy work (saving/analysis) on background threads.
Event handler registered twice → Always do -= then += in Start(), and unsubscribe in Stop() / Disconnect().

Summary

Implemented a dual-layer UI: image layer + HUD layer
HUD stays fixed in screen coordinates, making it stable during future zoom/pan
Displayed FPS, exposure, and gain, with FPS refreshed at ~10 Hz
Ready for extensions like ROI overlays and HUD-included capture with RenderTargetBitmap

2025-12-25

Implementing Live View in a WPF App for Basler Cameras (pylon SDK / C# / .NET 8)

In the previous article (Displaying a Basler Camera Snapshot in a WPF App), we captured a single frame and displayed it in a WPF Image. This time, we’ll implement a live camera preview inside a WPF window.

The live view uses an event-driven approach: we receive frames via ImageGrabbed, convert them to BitmapSource, and update an Image control. The update is performed safely via the Dispatcher to avoid blocking or crashing the UI thread.

Environment / Assumptions

Basler pylon Camera Software Suite (reference Basler.Pylon)
.NET 8 / WPF (Windows desktop)
Camera: acA2500-14gm (assumed Mono8) If you use a color camera, you must convert based on pixel format such as BGR8packed.

Goal

Live preview controlled by Connect / Start / Stop / Disconnect
Smooth updates by continuously refreshing a BitmapSource bound to an Image
Safe UI updates (always via the Dispatcher)

UI (XAML)

GUILiveSampleXAML

We add two buttons: Live (start preview) and Stop (stop preview).

<Window x:Class=”BaslerGUISample.MainWindow”
        xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
        xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
        xmlns:d=”http://schemas.microsoft.com/expression/blend/2008″
        xmlns:mc=”http://schemas.openxmlformats.org/markup-compatibility/2006″
        xmlns:local=”clr-namespace:BaslerGUISample”
        mc:Ignorable=”d”
        Title=”BaslerGUISample” Height=”450″ Width=”500″ Closing=”Window_Closing”>
    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition Height=”Auto”/>
            <RowDefinition Height=”*”/>
        </Grid.RowDefinitions>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
            <ColumnDefinition Width=”*”/>
        </Grid.ColumnDefinitions>
        <Grid Grid.Row=”0″ Grid.Column=”0″>
            <Button Content=”Connect” Width=”80″ Margin=”4,0″ Command=”{Binding ConnectCommand}”/>
        </Grid>
        <Grid Grid.Row=”0″ Grid.Column=”1″>
            <Button Content=”Disconnect” Width=”80″ Margin=”4,0″ Command=”{Binding DisconnectCommand}”/>
        </Grid>
        <Grid Grid.Row=”0″ Grid.Column=”2″>
            <Button Content=”Snap” Width=”80″ Margin=”4,0″ Command=”{Binding SnapCommand}”/>
        </Grid>
        <Grid Grid.Row=”0″ Grid.Column=”3″>
            <Button Content=”Live” Width=”80″ Margin=”4,0″ Command=”{Binding StartCommand}”/>
        </Grid>
        <Grid Grid.Row=”0″ Grid.Column=”4″>
            <Button Content=”Stop” Width=”80″ Margin=”4,0″ Command=”{Binding StopCommand}”/>
        </Grid>
        <Grid Grid.Row=”1″ Grid.Column=”0″ Grid.ColumnSpan=”5″>
            <Image x:Name=”PreviewImage” Source=”{Binding CurrentFrame}” Stretch=”Uniform”/>
        </Grid>
    </Grid>
</Window>

<Window x:Class=“BaslerGUISample.MainWindow”

xmlns=“http://schemas.microsoft.com/winfx/2006/xaml/presentation”

xmlns:x=“http://schemas.microsoft.com/winfx/2006/xaml”

xmlns:d=“http://schemas.microsoft.com/expression/blend/2008”

xmlns:mc=“http://schemas.openxmlformats.org/markup-compatibility/2006”

xmlns:local=“clr-namespace:BaslerGUISample”

mc:Ignorable=“d”

Title=“BaslerGUISample” Height=“450” Width=“500” Closing=“Window_Closing”>

<Grid>

<Grid.RowDefinitions>

</Grid.RowDefinitions>

<Grid.ColumnDefinitions>

</Grid.ColumnDefinitions>

</Grid>

</Grid>

</Grid>

</Grid>

</Grid>

</Grid>

</Window>

The code-behind is the same as in the previous article (set DataContext to the ViewModel and disconnect on closing).

Model Side (`BaslerCameraSample`)

If we reuse the previous StartGrabbing() implementation, frames keep piling up in ConcurrentQueue<(DateTime, IGrabResult)> _bufferedImageQueue, which is great for async saving but makes live view unnecessarily complex.

So, we add a simple streaming method dedicated to live display:

/// <summary>
/// Starts streaming without enqueuing frames into _bufferedImageQueue.
/// This method is specialized for live preview.
/// </summary>
public void StartLive()
{
    if (Camera == null || !IsConnected)
        throw new InvalidOperationException(“Camera is not connected.”);

    if (IsGrabbing)
        throw new InvalidOperationException(“Camera is already grabbing.”);

    // Set acquisition mode to continuous
    SetPLCameraParameter(PLCamera.AcquisitionMode, PLCamera.AcquisitionMode.Continuous);
    Camera.StreamGrabber.Start(GrabStrategy.OneByOne, GrabLoop.ProvidedByStreamGrabber);
}

/// <summary>

/// Starts streaming without enqueuing frames into _bufferedImageQueue.

/// This method is specialized for live preview.

/// </summary>

public void StartLive()

{

if (Camera == null || !IsConnected)

throw new InvalidOperationException(“Camera is not connected.”);

if (IsGrabbing)

throw new InvalidOperationException(“Camera is already grabbing.”);

// Set acquisition mode to continuous

SetPLCameraParameter(PLCamera.AcquisitionMode, PLCamera.AcquisitionMode.Continuous);

Camera.StreamGrabber.Start(GrabStrategy.OneByOne, GrabLoop.ProvidedByStreamGrabber);

}

Add/Remove event handlers

To keep the design flexible, we expose methods to attach/detach ImageGrabbed handlers:

/// <summary>
/// Adds an event handler that is called when an image is grabbed.
/// </summary>
public void AddImageGrabbedEventHandler(EventHandler<ImageGrabbedEventArgs> handler)
{
    if (Camera == null)
        return;
    Camera!.StreamGrabber!.ImageGrabbed += handler;
}

public void RemoveImageGrabbedEventHandler(EventHandler<ImageGrabbedEventArgs> handler)
{
    if (Camera == null)
        return;
    Camera.StreamGrabber!.ImageGrabbed -= handler;
}

/// <summary>

/// Adds an event handler that is called when an image is grabbed.

/// </summary>

public void AddImageGrabbedEventHandler(EventHandler<ImageGrabbedEventArgs> handler)

{

if (Camera == null)

return;

Camera!.StreamGrabber!.ImageGrabbed += handler;

}

public void RemoveImageGrabbedEventHandler(EventHandler<ImageGrabbedEventArgs> handler)

{

if (Camera == null)

return;

Camera.StreamGrabber!.ImageGrabbed -= handler;

}

ViewModel: Event-Driven Preview → Update `BitmapSource`

We continue using BaslerCameraSample, but now we update CurrentFrame whenever ImageGrabbed fires.

If performance becomes an issue, consider using WriteableBitmap rather than creating a new BitmapSource every time.

using Basler.Pylon; // ImageGrabbedEventArgs
using BaslerSamples; // Camera control library
// The author uses BindableBase / DelegateCommand from a Common namespace.
// Please adapt to your environment.
// using Common;
using System;
using System.Windows;
using System.Windows.Media.Imaging;
using Application = System.Windows.Application;

namespace BaslerGUISample.ViewModels
{
    public partial class MainViewModel : BindableBase
    {
        private readonly BaslerCameraSample _cameraService = new();

        public MainViewModel()
        {
            // ConnectCommand, DisconnectCommand, SnapCommand are the same as the previous article.
            StartCommand = new DelegateCommand(Start, () => IsConnected && !IsGrabbing);
            StopCommand  = new DelegateCommand(Stop,  () => IsConnected && IsGrabbing);
        }

        private bool _isConnected;
        public bool IsConnected
        {
            get => _isConnected;
            set => SetProperty(ref _isConnected, value);
        }

        /// <summary>
        /// Indicates whether the camera is currently grabbing frames.
        /// </summary>
        private bool _isGrabbing;
        public bool IsGrabbing
        {
            get => _isGrabbing;
            set => SetProperty(ref _isGrabbing, value);
        }

        private BitmapSource? _currentFrame;
        public BitmapSource? CurrentFrame
        {
            get => _currentFrame;
            set => SetProperty(ref _currentFrame, value);
        }

        public void Start()
        {
            if (IsConnected == false) return;

            // Register handler (remove first to avoid duplicates)
            _cameraService.RemoveImageGrabbedEventHandler(OnImageGrabbed);
            _cameraService.AddImageGrabbedEventHandler(OnImageGrabbed);

            _cameraService.StartLive();
            IsGrabbing = _cameraService.IsGrabbing;
        }

        public void Stop()
        {
            if (_cameraService.IsConnected == false) return;

            _cameraService.StopGrabbing();

            // Unregister handler
            _cameraService.RemoveImageGrabbedEventHandler(OnImageGrabbed);
            IsGrabbing = _cameraService.IsGrabbing;
        }

        /// <summary>
        /// Called when a frame is received.
        /// </summary>
        private async void OnImageGrabbed(object? sender, ImageGrabbedEventArgs e)
        {
            IsGrabbing = _cameraService.IsGrabbing;

            // Clone because we don’t control when the original GrabResult gets disposed.
            using IGrabResult result = e.GrabResult.Clone();

            try
            {
                // Always update the image on the UI thread.
                await Application.Current.Dispatcher.InvokeAsync(() =>
                {
                    var frame = BaslerCameraSample.ConvertGrabResultToBitmap(result);
                    CurrentFrame = frame;
                });
            }
            catch (InvalidOperationException ex)
            {
                MessageBox.Show($”Failed to grab. Error: {ex.Message}”);
            }
        }
    }
}

using Basler.Pylon; // ImageGrabbedEventArgs

using BaslerSamples; // Camera control library

// The author uses BindableBase / DelegateCommand from a Common namespace.

// Please adapt to your environment.

// using Common;

using System;

using System.Windows;

using System.Windows.Media.Imaging;

using Application = System.Windows.Application;

namespace BaslerGUISample.ViewModels

{

public partial class MainViewModel : BindableBase

{

private readonly BaslerCameraSample _cameraService = new();

public MainViewModel()

{

// ConnectCommand, DisconnectCommand, SnapCommand are the same as the previous article.

StartCommand = new DelegateCommand(Start, () => IsConnected && !IsGrabbing);

StopCommand = new DelegateCommand(Stop, () => IsConnected && IsGrabbing);

}

private bool _isConnected;

public bool IsConnected

{

get => _isConnected;

set => SetProperty(ref _isConnected, value);

}

/// <summary>

/// Indicates whether the camera is currently grabbing frames.

/// </summary>

private bool _isGrabbing;

public bool IsGrabbing

{

get => _isGrabbing;

set => SetProperty(ref _isGrabbing, value);

}

private BitmapSource? _currentFrame;

public BitmapSource? CurrentFrame

{

get => _currentFrame;

set => SetProperty(ref _currentFrame, value);

}

public void Start()

{

if (IsConnected == false) return;

// Register handler (remove first to avoid duplicates)

_cameraService.RemoveImageGrabbedEventHandler(OnImageGrabbed);

_cameraService.AddImageGrabbedEventHandler(OnImageGrabbed);

_cameraService.StartLive();

IsGrabbing = _cameraService.IsGrabbing;

}

public void Stop()

{

if (_cameraService.IsConnected == false) return;

_cameraService.StopGrabbing();

// Unregister handler

_cameraService.RemoveImageGrabbedEventHandler(OnImageGrabbed);

IsGrabbing = _cameraService.IsGrabbing;

}

/// <summary>

/// Called when a frame is received.

/// </summary>

private async void OnImageGrabbed(object? sender, ImageGrabbedEventArgs e)

{

IsGrabbing = _cameraService.IsGrabbing;

// Clone because we don’t control when the original GrabResult gets disposed.

using IGrabResult result = e.GrabResult.Clone();

try

{

// Always update the image on the UI thread.

await Application.Current.Dispatcher.InvokeAsync(() =>

{

var frame = BaslerCameraSample.ConvertGrabResultToBitmap(result);

CurrentFrame = frame;

});

}

catch (InvalidOperationException ex)

{

MessageBox.Show($“Failed to grab. Error: {ex.Message}”);

}

Example Run

Click Connect, then Live to start continuous preview. Click Stop to stop streaming.

GUILiveSample

Common Pitfalls & Fixes

Colors look wrong (color cameras) Match conversion to the camera’s pixel format (e.g., PixelType.BGR8packed and PixelFormats.Bgr24).
Event handler registered twice Remove (-=) once before adding (+=), as shown in Start().
Rendering can’t keep up with camera FPS If the camera FPS is high, you often don’t need to render every frame. You can drop frames by using Monitor.TryEnter or Interlocked.Exchange to avoid overlapping UI updates.

Summary

Implemented smooth live view using pylon’s event-driven acquisition
Updated WPF UI safely through the Dispatcher
The structure is flexible and works well with ROI and resolution changes
Combining live preview with triggers is already valuable for many lab/inspection setups

Next: HUD Overlay

In the next article, we will overlay a HUD showing values such as FPS, exposure time, and gain on top of the live preview.

2025-12-17

タグ: WPF

Implementing Background Subtraction (Motion Detection) in WPF with Basler pylon SDK + OpenCV (C# / .NET 8)

✅ Environment

Implementation Overview

🧩 XAML (Minimal UI)

🔧 Core Code (Background Subtraction Pipeline)

Code-Behind

Example Run

Tuning Tips

Common Pitfalls

Summary

Zoom & Pan for WPF Live View Using TransformGroup (C# / .NET 8)

Environment / Prerequisites

Goals

XAML: Two Layers + ROI Layer (Shared Transform)

HalfConverter (For Center Crosshair)

Code-Behind: Zoom / Pan

ROI Handling and Applying It in Real Apps

Example Screens

Summary

Overlaying a HUD on Basler Live View in WPF (FPS, Exposure, Gain) — Dual-Layer UI Layout

Environment / Assumptions

Goal

XAML: Dual Layer (with Optional ROI Layer)

ViewModel: Throttled FPS Updates + Exposure/Gain HUD

Example Output

Common Pitfalls & Fixes

Summary

Implementing Live View in a WPF App for Basler Cameras (pylon SDK / C# / .NET 8)

Environment / Assumptions

Goal

UI (XAML)

Model Side (BaslerCameraSample)

Add/Remove event handlers

ViewModel: Event-Driven Preview → Update BitmapSource

Example Run

Common Pitfalls & Fixes

Summary

Next: HUD Overlay

Model Side (`BaslerCameraSample`)

ViewModel: Event-Driven Preview → Update `BitmapSource`