Skip to content

Panorama Stitching: Stitcher.Stitch crashes with cv::gemm assertion (CV_64FC2 type mismatch) after upgrading to 5.0.0.20260712 #12

Description

@shimat

Summary

After upgrading to OpenCvSharp5/OpenCvSharp5.runtime.wasm 5.0.0.20260712 (which includes the OpenCL fix from opencvsharp#2041), the /samples/aruco page's OpenCL/UMat crash is confirmed resolved. However, /samples/stitching's Cv2.Stitcher.Stitch now fails with a different crash - no longer the OpenCL probe, but an assertion failure deep in the bundle adjustment step:

Uncaught (in promise) RuntimeError
cv::Exception: OpenCV(5.0.0) /home/runner/work/opencvsharp/opencvsharp/opencv-5.0.0/modules/core/src/matmul.simd.hpp:1053:
error: (-215:Assertion failed) type == CV_64FC2 in function 'gemmImpl'

    at cv::error(cv::Exception const&)
    at cv::cpu_baseline::gemmImpl(cv::Mat, cv::Mat, double, cv::Mat, double, cv::Mat, int)
    at cv::cpu_baseline::callGemmImpl<double>(...)
    at cv::gemm(cv::_InputArray const&, cv::_InputArray const&, double, cv::_InputArray const&, double, cv::_OutputArray const&, int)
    at cv::mulTransposed(cv::_InputArray const&, cv::_OutputArray const&, bool, cv::_InputArray const&, double, int)
    at cv::LevMarqDenseLinearBackend::calcFunc(double&, bool, bool)
    at cv::detail::LevMarqBase::optimize()

followed immediately by a second, unrecoverable trap:

RuntimeError: unreachable
    at __trap
    at abort
    at assert
    at runtimeKeepalivePop

Where this happens

PanoramaStitching.razor's StitchAsync:

using var stitcher = Stitcher.Create(Stitcher.Mode.Panorama);
using var pano = new Mat();
var result = stitcher.Stitch(new[] { leftMat, rightMat }, pano);

leftMat/rightMat are two crops of the same 256x256 Mandrill.bmp (180px and 156px wide respectively) sharing an ~80px overlapping band - a deliberately small, synthetic "two photos of the same scene" pair, standing in for a real photo pair a user hasn't uploaded yet.

Question / ask

I don't yet know whether this is:

  • a genuine bug in OpenCV's LevMarqDenseLinearBackend/bundle-adjustment code (a real type mismatch building the parameter matrix passed into gemm), which would need to be reported to opencv/opencv instead, or
  • a usage issue on my end - e.g. the crop pair being too small / too little true parallax for the default bundle adjuster, or Stitcher.Mode.Panorama/the default bundle adjuster needing different settings for such small inputs

Filing here first since I'm not confident yet it's a core algorithm bug rather than something about how this sample is calling Stitcher. If it turns out to reproduce with a normal-sized, real photo pair (not just this synthetic small-crop case), I'll follow up with a report against opencv/opencv directly. Any pointers on which it is - or whether this reproduces outside wasm at all - would be appreciated.

Environment

  • OpenCvSharp5 / OpenCvSharp5.runtime.wasm 5.0.0.20260712
  • Confirmed the OpenCL/UMat crash (opencvsharp#2037) is fixed as of this version - this is a separate, newly-surfaced issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions