错误“对于非平面校准装置，必须在函数‘cvCalibrateCamera2Internal’中指定初始固有矩阵”

我遇到的错误的完整跟踪：

在stereo_calibrate中 ret，cameraMatrix1，distCoeffs1，cameraMatrix2，distCoeffs2，R，T，E，F，perViewErrors，_，_= cv2.stereoCalibrateExtended（ cv2.error: OpenCV(4.10.0) /io/opencv/modules/calib3d/src/calibration.cpp:1682: error: (-5:Bad argument) 对于非平面校准装置，必须在中指定初始内在矩阵函数'cvCalibrateCamera2Internal'

这是我的代码：

def stereo_calibrate(img, correspondences_left, correspondences_right, camera_matrix_L, camera_matrix_R, baseline=4.1, dist_coeffs=None):
    if dist_coeffs is None:
        dist_coeffs = np.zeros((4, 1))

    # Extract image points and object points from correspondences
    image_points_left = correspondences_left[:, :2].astype(np.float32)
    image_points_right = correspondences_right[:, :2].astype(np.float32)
    object_points = correspondences_left[:, 2:].astype(np.float32)  # Assuming object points are the same for both sets

    rvec_L, tvec_L = np.array([0., 0., 0.]), np.array([0, 0., 0.])
    rvec_R, tvec_R = np.array([-0.00637872, 0.00102092, 0.000673804]), np.array([0., 0., 0.])

    # Convert rotation vectors to rotation matrices
    R_L, _ = cv2.Rodrigues(rvec_L)
    R_R, _ = cv2.Rodrigues(rvec_R)
    print("after converting via Rodrigues:", R_L, R_R)

    # Initialize rotation and translation vectors
    R_init = R_R @ R_L.T
    # T_init = tvec_R - R_init @ tvec_L
    T_init = np.array([0., 0., 0.])
    print("R_init", R_init)
    print("T_init", T_init)

    # Stereo calibration to refine the extrinsic parameters
    ret, cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, R, T, E, F, perViewErrors, _, _= cv2.stereoCalibrateExtended(
        [object_points], [image_points_left], [image_points_right],
        camera_matrix_L, dist_coeffs, camera_matrix_R, dist_coeffs,
        imageSize=(img.shape[1], img.shape[0]),  # Correct image size as (width, height)
        R=R_init, T=T_init,
        criteria=(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 1e-6),
        flags=cv2.CALIB_USE_EXTRINSIC_GUESS
    )
    print("ret", ret)
    if ret:
        print("Stereo Calibration Successful")
        print("Rotation Matrix between the cameras:")
        print(R)
        print("Translation Vector between the cameras:")
        print(T)
        print("dist_coeffs1", distCoeffs1)
        print("dist_coeffs2", distCoeffs2)
        print(rvec_L)
        print(tvec_L)
    else:
        raise ValueError("stereoCalibrate failed to find a solution")

    # Reproject object points to both cameras
    reprojected_points_left, _ = cv2.projectPoints(object_points, rvec_L, tvec_L, camera_matrix_L, distCoeffs1)
    reprojected_points_right, _ = cv2.projectPoints(object_points, R, T, camera_matrix_R, distCoeffs2)
    
    return R, T, reprojected_points_left, reprojected_points_right

correspondences_left = np.array([
    [671.0889955686854, 193.80354505169868, 2.4492935982947064e-16, -4.0, 11],
...
])

correspondences_right = np.array([
    [436.2245592329106, 193.79399938137954, 2.4492935982947064e-16, -4.0, 11],
...
])

# Camera intrinsic parameters, replace with your actual camera matrix
camera_matrix_L = np.array([
    [1018.9, 0, 601.447],
    [0, 1018.9, 517.462],
    [0, 0, 1]
], dtype=np.float32)

camera_matrix_R = np.array([
    [1018.9, 0, 690.392],
    [0, 1018.9, 517.462],
    [0, 0, 1]
], dtype=np.float32)

# calling the function
R, T, reprojected_points_left, reprojected_points_right = stereo_calibrate(img_left, correspondences_left, correspondences_right, camera_matrix_L, camera_matrix_R)

这个错误真的让我困惑，因为我显然正在传递初始相机内在矩阵。为什么会发生这个错误？

The issue lies in how you are calling the cv2.stereoCalibrateExtended function and what it expects when you provide an initial guess for the extrinsic parameters.

Let's break down the problem and the solution:

Problem:

• Non-Planar Calibration: The error message "For non-planar calibration rigs, it is required to specify initial intrinsic matrices in function 'cvCalibrateCamera2Internal'" tells us that OpenCV assumes you are using a non-planar calibration pattern (like a 3D target). Non-planar calibration needs more information to initialize the optimization process effectively.
• Initial Guess Misinterpretation: When you provide R and T to cv2.stereoCalibrateExtended with cv2.CALIB_USE_EXTRINSIC_GUESS , you're telling it to use these as initial guesses for the stereo extrinsic parameters (relationship between the cameras). However, the underlying cvCalibrateCamera2Internal function, which handles the per-camera calibration, needs initial guesses for the intrinsic parameters of each camera if you're not using a planar target.

Solution:

To resolve this, you have two main options:

1. Use cv2.CALIB_FIX_INTRINSIC :

2. If you are confident that your camera_matrix_L and camera_matrix_R are accurate and don't need to be refined during stereo calibration, you can tell OpenCV to keep them fixed:

   python flags=cv2.CALIB_USE_EXTRINSIC_GUESS | cv2.CALIB_FIX_INTRINSIC

3. Important: This assumes your camera matrices are very good. If they are off, your stereo calibration will be inaccurate.

4. Provide Initial Intrinsic Guesses (If Necessary):

5. If you want to refine the intrinsic parameters slightly during stereo calibration, you can provide "good enough" initial guesses. Often, using the matrices you already have is a reasonable starting point.

   python ret, cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, R, T, E, F, perViewErrors, _, _= cv2.stereoCalibrateExtended( [object_points], [image_points_left], [image_points_right], camera_matrix_L, dist_coeffs, camera_matrix_R, dist_coeffs, imageSize=(img.shape[1], img.shape[0]), R=R_init, T=T_init, criteria=(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 1e-6), flags=cv2.CALIB_USE_EXTRINSIC_GUESS | cv2.CALIB_USE_INTRINSIC_GUESS )

Additional Notes:

• Calibration Target: The error message strongly suggests you are using a non-planar calibration pattern. If you are using a planar target, you can usually omit providing initial intrinsic guesses (unless you have a specific reason).
• Accuracy: Accurate stereo calibration depends heavily on good input data (images, correspondences, calibration pattern). Make sure your calibration setup and image acquisition process are solid.

Let me know if you have any more questions!