Image Processing in the Browser

~20 min Intermediate

Use BptImageAnalysis for client-side edge detection (Sobel, Canny, Contour, Skeleton) and BptAnimationEffect for GPU-accelerated visuals. Everything runs in the user's browser.

Home / Learning / Image Processing in the Browser
No server roundtrip Both components do all their work on the client. BptImageAnalysis uses Canvas 2D pixel operations; BptAnimationEffect uses WebGL 1 (with a Canvas 2D fallback). Your server never sees the image data — useful for privacy-sensitive workflows like document or ID scanning.

Part 1 — BptImageAnalysis

Step 1: Render an image with detection layers

The component renders an <img>, then overlays one or more detection layers on top. Each layer is independently toggleable, has its own parameters, and is painted on a separate <canvas> stacked above the image.

@page "/scan" @using Bpt.Components.Image <BptImageAnalysis Src="/images/sample.jpg" Width="640px" LineDetectEnabled="true" LineDetectSensitivity="60" LineDetectColor="#00ff88" />

That single line turns on the Sobel-based line detector (BPT's friendly name is "LineDetect"). The sensitivity is a 0–100 threshold — lower values pick up faint edges, higher values keep only the strongest ones.

Step 2: Pick the right detector

BPT ships four detection algorithms. Each is a different trade between speed, accuracy, and what kind of structure it picks up.

AlgorithmWhat it findsSpeedBest for
LineDetect (Sobel) Strong gradients — sharp edges in any direction. Fastest. ~10 ms on a 1280×720 frame. General-purpose edge highlighting, quick previews.
CannyEdgeDetect Single-pixel-wide edges via Gaussian blur + NMS + hysteresis. Slower (~30 ms). Three threshold parameters. Document edge detection, line art, OCR preprocessing.
ContourFinder Closed shapes — outlines you could fill. Medium. Shape recognition, blob detection, ROI extraction.
Skeletonize One-pixel-wide "skeleton" through the centre of shapes. Medium-slow (iterative thinning). Handwriting, biological imaging, road centerlines.

You can enable all four simultaneously — each renders into its own canvas, in its own color. That's the basis of the live demo, where every layer toggles independently.

Step 3: Canny in depth

Canny is the workhorse for clean, single-pixel edges. It has three knobs:

ParameterEffectTuning
CannyEdgeDetectSigmaGaussian blur kernel size before edge detection.Larger sigma = more noise rejection but loses fine details. Default 1.4 works for most photos; bump to 2.0–2.5 for noisy/scanned input.
CannyEdgeDetectLowThresholdMinimum gradient strength to seed an edge candidate.Lower = more edges picked up. Typical range 10–40.
CannyEdgeDetectHighThresholdStrength required to start a new edge segment (hysteresis).Usually 2–3× the low threshold. Default 20/50 is a balanced starting point.
<BptImageAnalysis Src="@_imageSrc" CannyEdgeDetectEnabled="true" CannyEdgeDetectSigma="2.0f" CannyEdgeDetectLowThreshold="15" CannyEdgeDetectHighThreshold="40" CannyEdgeDetectColor="#ff00ff" CannyEdgeDetectAntiAlias="3" />

The CannyEdgeDetectAntiAlias parameter blurs the resulting edge field for smoother rendering. Canny edges are by definition single-pixel-wide, so a value of 2–4 softens them without losing structural fidelity.

Step 4: Capturing snapshots

For workflows that need a "save the detected edges" step (think: a scanning app that uploads a clean version), subscribe to OnPatternSnapshot. It fires with an AnalysisPatternSnapshotData object containing the rendered overlay as a base64 data URL.

<BptImageAnalysis Src="@_src" ContourFinderEnabled="true" OnPatternSnapshot="OnSnapshot" /> @code { private string? _capturedDataUrl; private void OnSnapshot(AnalysisPatternSnapshotData data) { _capturedDataUrl = data.ImageDataUrl; // Optionally upload, store, or hand off to another component. } }

Part 2 — BptAnimationEffect

Step 5: GPU-accelerated visuals

BptAnimationEffect wraps a child fragment and overlays a GPU-driven effect on top. It's a single component with 17+ effects selected via the Effect enum.

CategoryEffectsUse case
Ambient WaterRipple, SeaWater, Rain Background motion on landing pages, login screens.
Entrance ZoomIn, SlideInTop/Bottom/Left/Right, Instant Modal/dialog reveals, route transitions.
Exit ZoomOut, SlideOutTop/Bottom/Left/Right Dismiss animations paired with an entrance.
Visual Perspective, Refraction Hero images, product showcases.

Step 6: A water-ripple hero

<BptAnimationEffect Effect="BptEffect.WaterRipple" Width="100%" Height="400px" Color="rgba(120, 180, 255, 0.35)" Speed="2.5" Spread="80"> <div class="hero-content"> <h1>Welcome aboard</h1> <p>Sign in to continue.</p> </div> </BptAnimationEffect>

The ChildContent renders on top of the effect canvas, with normal stacking — your text and buttons stay interactive. Color tints the effect, Speed scales the animation rate, and Spread controls ripple radius for the water variants.

Step 7: WebGL with a Canvas 2D fallback

Under the hood, BptAnimationEffect tries WebGL 1 first. If the browser refuses (lost context, no GPU, old hardware), it falls back to a Canvas 2D version that produces the same effect at lower fidelity. You don't write fallback code — it just works.

Don't dispose canvases on toggles WebGL contexts bind permanently to their canvas element. If you conditionally render BptAnimationEffect with @if, the canvas is destroyed on toggle and a new one is created — usually safe, but rapid toggling can trigger loseContext events the component recovers from. For very high-frequency mounting/unmounting, wrap in a stable @key or hide via CSS instead.

Step 8: Composing detection + animation

The two components compose cleanly — wrap BptImageAnalysis in a BptAnimationEffect for a "ripple over the scanned document" preview. Because BptAnimationEffect uses WebGL on a transparent overlay, the underlying analysis canvases still render and respond to parameter changes.

<BptAnimationEffect Effect="BptEffect.Refraction" Speed="1.5"> <BptImageAnalysis Src="@_src" LineDetectEnabled="true" CannyEdgeDetectEnabled="true" /> </BptAnimationEffect>

Common gotchas

  • CORS on image sources. If Src points to a different domain, the browser blocks pixel reads — and the detection layers come back blank. Either host the image on the same origin, or serve it with Access-Control-Allow-Origin: *.
  • Anti-alias on small details. Pushing LineDetectAntiAlias above 4 starts to dissolve thin features. If you're losing structure, lower the value first before reaching for other knobs.
  • Animation on low-power devices. WaterRipple and SeaWater are GPU-bound and can drop frames on integrated graphics. The fallback path is automatic, but you may want to disable the effect entirely on phones — check navigator.deviceMemory from JS interop and skip animations on devices with ≤ 2 GB.
  • Canvas 2D path skips Refraction. Refraction and Perspective rely on per-pixel WebGL shaders. The 2D fallback renders the wrapped content with no effect — a clean degradation rather than a broken display.

An unhandled error has occurred. Reload 🗙

Rejoining the server...

Rejoin failed... trying again in seconds.

Failed to rejoin.
Please retry or reload the page.

The session has been paused by the server.

Failed to resume the session.
Please reload the page.