立体视差

概述

立体视差应用程序接收左右立体图像对，并返回它们之间的视差，视差是图像深度的函数。结果将保存为图像文件到磁盘。如果可用，它还将输出相应的置信度图。

说明

命令行参数为

<backend> <left image> <right image>

其中

• backend：可以是 cuda、ofa 或 ofa-pva-vic；它定义了将执行处理的后端。 ofa-pva-vic 和 cuda 允许除了视差之外还输出置信度图。
• left image：校正后的立体图像对的左输入图像，它接受 png、jpeg 以及可能的其他格式。
• right image：立体图像对的右输入图像。

这是一个例子

• C++

  ./vpi_sample_02_stereo_disparity cuda ../assets/chair_stereo_left.png ../assets/chair_stereo_right.png

• Python

  python3 main.py cuda ../assets/chair_stereo_left.png ../assets/chair_stereo_right.png

这是使用 CUDA 后端和提供的示例图像。您可以尝试使用其他立体图像对，但要遵守算法施加的约束。

此示例的 Python 版本还允许设置各种附加参数，以及附加的输入图像扩展名和打开详细模式。以下命令行参数可以传递给 Python 示例

• Python

  python3 main.py <backend> <left image> <right image> --width W --height H --downscale D --window_size WIN
  --skip_confidence --conf_threshold T --conf_type absolute/relative -p1 P1 -p2 P2 --p2_alpha P2alpha
  --uniqueness U --skip_diagonal --num_passes N --min_disparity MIN --max_disparity MAX --output_mode 0/1/2
  -v/--verbose

  其中附加的可选参数为
• width：当传递 ".raw" 输入图像时，设置宽度 W
• height：当传递 ".raw" 输入图像时，设置高度 H
• downscale：设置输出的缩小比例因子为 D
• window_size：设置中值滤波器窗口大小为 WIN
• skip_confidence：避免计算置信度并将其用作掩码
• conf_threshold：设置置信度阈值为 T
• conf_type：设置置信度类型为 absolute 或 relative
• p1：设置 p1 惩罚为 P1
• p2：设置 p2 惩罚为 P2
• p2_alpha：设置 p2Alpha 自适应惩罚为 P2alpha
• uniqueness：设置唯一性为 U
• skip_diagonal：避免在 CUDA 或 OFA 后端中使用对角路径
• num_passes：在 OFA 后端中设置通过次数 N
• min_disparity：在 CUDA 后端中设置最小视差 MIN
• max_disparity：在后端中设置最大视差 MAX
• output_mode：0 表示彩色输出，1 表示灰度输出，2 表示原始二进制输出
• verbose：打开详细模式。要详细了解与立体视差算法相关的每个附加参数，请阅读相应的文档。

结果

左输入图像	右输入图像
立体视差	置信度图

源代码

为了方便起见，这里是代码，它也安装在 samples 目录中。

语言 C++ Python

import sys
import vpi
import numpy as np
from PIL import Image
from argparse import ArgumentParser
import cv2


def read_raw_file(fpath, resize_to=None, verbose=False)
 try
 if verbose
 print(f'I Reading: {fpath}', end=' ', flush=True)
 f = open(fpath, 'rb')
 np_arr = np.fromfile(f, dtype=np.uint16, count=-1)
 f.close()
 if verbose
 print(f'done!\nI Raw array: shape: {np_arr.shape} dtype: {np_arr.dtype}')
 if resize_to is not None
 np_arr = np_arr.reshape(resize_to, order='C')
 if verbose
 print(f'I Reshaped array: shape: {np_arr.shape} dtype: {np_arr.dtype}')
 pil_img = Image.fromarray(np_arr, mode="I;16L")
 return pil_img
 except
 raise ValueError(f'E Cannot process raw input: {fpath}')


def process_arguments()
 parser = ArgumentParser()

 parser.add_argument('backend', choices=['cuda','ofa','ofa-pva-vic'],
 help='Backend to be used for processing')
 parser.add_argument('left', help='Rectified left input image from a stereo pair')
 parser.add_argument('right', help='Rectified right input image from a stereo pair')
 parser.add_argument('--width', default=-1, type=int, help='Input width for raw input files')
 parser.add_argument('--height', default=-1, type=int, help='Input height for raw input files')
 parser.add_argument('--downscale', default=1, type=int, help='Output downscale factor')
 parser.add_argument('--window_size', default=5, type=int, help='Median filter window size')
 parser.add_argument('--skip_confidence', default=False, action='store_true', help='Do not calculate confidence')
 parser.add_argument('--conf_threshold', default=32767, type=int, help='Confidence threshold')
 parser.add_argument('--conf_type', default='best', choices=['best', 'absolute', 'relative', 'inference'],
 help='Computation type to produce the confidence output. Default will pick best option given backend.')
 parser.add_argument('-p1', default=3, type=int, help='Penalty P1 on small disparities')
 parser.add_argument('-p2', default=48, type=int, help='Penalty P2 on large disparities')
 parser.add_argument('--p2_alpha', default=0, type=int, help='Alpha for adaptive P2 Penalty')
 parser.add_argument('--uniqueness', default=-1, type=float, help='Uniqueness ratio')
 parser.add_argument('--skip_diagonal', default=False, action='store_true', help='Do not use diagonal paths')
 parser.add_argument('--num_passes', default=3, type=int, help='Number of passes')
 parser.add_argument('--min_disparity', default=0, type=int, help='Minimum disparity')
 parser.add_argument('--max_disparity', default=256, type=int, help='Maximum disparity')
 parser.add_argument('--output_mode', default=0, type=int, help='0: color; 1: grayscale; 2: raw binary')
 parser.add_argument('-v', '--verbose', default=False, action='store_true', help='Verbose mode')

 return parser.parse_args()


def main()
 args = process_arguments()

 scale = 1 # pixel value scaling factor when loading input

 if args.backend == 'cuda'
 backend = vpi.Backend.CUDA
 elif args.backend == 'ofa'
 backend = vpi.Backend.OFA
 elif args.backend == 'ofa-pva-vic'
 backend = vpi.Backend.OFA|vpi.Backend.PVA|vpi.Backend.VIC
 else
 raise ValueError(f'E Invalid backend: {args.backend}')

 conftype = None
 if args.conf_type == 'best'
 conftype = vpi.ConfidenceType.INFERENCE if args.backend == 'ofa-pva-vic' else vpi.ConfidenceType.ABSOLUTE
 elif args.conf_type == 'absolute'
 conftype = vpi.ConfidenceType.ABSOLUTE
 elif args.conf_type == 'relative'
 conftype = vpi.ConfidenceType.RELATIVE
 elif args.conf_type == 'inference'
 conftype = vpi.ConfidenceType.INFERENCE
 else
 raise ValueError(f'E Invalid confidence type: {args.conf_type}')

 minDisparity = args.min_disparity
 maxDisparity = args.max_disparity
 includeDiagonals = not args.skip_diagonal
 numPasses = args.num_passes
 calcConf = not args.skip_confidence
 downscale = args.downscale
 windowSize = args.window_size
 quality = 6

 if args.verbose
 print(f'I Backend: {backend}\nI Left image: {args.left}\nI Right image: {args.right}\n'
 f'I Disparities (min, max): {(minDisparity, maxDisparity)}\n'
 f'I Input scale factor: {scale}\nI Output downscale factor: {downscale}\n'
 f'I Window size: {windowSize}\nI Quality: {quality}\n'
 f'I Calculate confidence: {calcConf}\nI Confidence threshold: {args.conf_threshold}\n'
 f'I Confidence type: {conftype}\nI Uniqueness ratio: {args.uniqueness}\n'
 f'I Penalty P1: {args.p1}\nI Penalty P2: {args.p2}\nI Adaptive P2 alpha: {args.p2_alpha}\n'
 f'I Include diagonals: {includeDiagonals}\nI Number of passes: {numPasses}\n'
 f'I Output mode: {args.output_mode}\nI Verbose: {args.verbose}\n'
 , end='', flush=True)

 if 'raw' in args.left
 pil_left = read_raw_file(args.left, resize_to=[args.height, args.width], verbose=args.verbose)
 np_left = np.asarray(pil_left)
 else
 try
 pil_left = Image.open(args.left)
 if pil_left.mode == 'I'
 np_left = np.asarray(pil_left).astype(np.int16)
 else
 np_left = np.asarray(pil_left)
 except
 raise ValueError(f'E Cannot open left input image: {args.left}')

 if 'raw' in args.right
 pil_right = read_raw_file(args.right, resize_to=[args.height, args.width], verbose=args.verbose)
 np_right = np.asarray(pil_right)
 else
 try
 pil_right = Image.open(args.right)
 if pil_right.mode == 'I'
 np_right = np.asarray(pil_right).astype(np.int16)
 else
 np_right = np.asarray(pil_right)
 except
 raise ValueError(f'E Cannot open right input image: {args.right}')

 # Streams for left and right independent pre-processing
 streamLeft = vpi.Stream()
 streamRight = vpi.Stream()

 # Load input into a vpi.Image and convert it to grayscale, 16bpp
 with vpi.Backend.CUDA
 with streamLeft
 left = vpi.asimage(np_left).convert(vpi.Format.Y16_ER, scale=scale)
 with streamRight
 right = vpi.asimage(np_right).convert(vpi.Format.Y16_ER, scale=scale)

 # Preprocess input
 # Block linear format is needed for ofa backends
 # We use VIC backend for the format conversion because it is low power
 if args.backend in {'ofa-pva-vic', 'ofa'}
 if args.verbose
 print(f'W {args.backend} forces to convert input images to block linear', flush=True)
 with vpi.Backend.VIC
 with streamLeft
 left = left.convert(vpi.Format.Y16_ER_BL)
 with streamRight
 right = right.convert(vpi.Format.Y16_ER_BL)

 if args.verbose
 print(f'I Input left image: {left.size} {left.format}\n'
 f'I Input right image: {right.size} {right.format}', flush=True)

 confidenceU16 = None

 if calcConf
 if args.backend not in {'cuda', 'ofa-pva-vic'}
 # Only CUDA and OFA-PVA-VIC support confidence map
 calcConf = False
 if args.verbose
 print(f'W {args.backend} does not allow to calculate confidence', flush=True)


 outWidth = (left.size[0] + downscale - 1) // downscale
 outHeight = (left.size[1] + downscale - 1) // downscale

 if calcConf
 confidenceU16 = vpi.Image((outWidth, outHeight), vpi.Format.U16)

 # Use stream left to consolidate actual stereo processing
 streamStereo = streamLeft

 if args.backend == 'ofa-pva-vic' and maxDisparity not in {128, 256}
 maxDisparity = 128 if (maxDisparity // 128) < 1 else 256
 if args.verbose
 print(f'W {args.backend} only supports 128 or 256 maxDisparity. Overriding to {maxDisparity}', flush=True)

 if args.verbose
 if 'ofa' not in args.backend
 print('W Ignoring P2 alpha and number of passes since not an OFA backend', flush=True)
 if args.backend != 'cuda'
 print('W Ignoring uniqueness since not a CUDA backend', flush=True)
 print('I Estimating stereo disparity ... ', end='', flush=True)

 # Estimate stereo disparity.
 with streamStereo, backend
 disparityS16 = vpi.stereodisp(left, right, downscale=downscale, out_confmap=confidenceU16,
 window=windowSize, maxdisp=maxDisparity, confthreshold=args.conf_threshold,
 quality=quality, conftype=conftype, mindisp=minDisparity,
 p1=args.p1, p2=args.p2, p2alpha=args.p2_alpha, uniqueness=args.uniqueness,
 includediagonals=includeDiagonals, numpasses=numPasses)

 if args.verbose
 print('done!\nI Post-processing ... ', end='', flush=True)

 # Postprocess results and save them to disk
 with streamStereo, vpi.Backend.CUDA
 # Some backends outputs disparities in block-linear format, we must convert them to
 # pitch-linear for consistency with other backends.
 if disparityS16.format == vpi.Format.S16_BL
 disparityS16 = disparityS16.convert(vpi.Format.S16, backend=vpi.Backend.VIC)

 # Scale disparity and confidence map so that values like between 0 and 255.

 # Disparities are in Q10.5 format, so to map it to float, it gets
 # divided by 32. Then the resulting disparity range, from 0 to
 # stereo.maxDisparity gets mapped to 0-255 for proper output.
 # Copy disparity values back to the CPU.
 disparityU8 = disparityS16.convert(vpi.Format.U8, scale=255.0/(32*maxDisparity)).cpu()

 # Apply JET colormap to turn the disparities into color, reddish hues
 # represent objects closer to the camera, blueish are farther away.
 disparityColor = cv2.applyColorMap(disparityU8, cv2.COLORMAP_JET)

 # Converts to RGB for output with PIL.
 disparityColor = cv2.cvtColor(disparityColor, cv2.COLOR_BGR2RGB)

 if calcConf
 confidenceU8 = confidenceU16.convert(vpi.Format.U8, scale=255.0/65535).cpu()

 # When pixel confidence is 0, its color in the disparity is black.
 mask = cv2.threshold(confidenceU8, 1, 255, cv2.THRESH_BINARY)[1]
 mask = cv2.cvtColor(mask, cv2.COLOR_GRAY2BGR)
 disparityColor = cv2.bitwise_and(disparityColor, mask)

 fext = '.raw' if args.output_mode == 2 else '.png'

 disparity_fname = f'disparity_python{sys.version_info[0]}_{args.backend}' + fext
 confidence_fname = f'confidence_python{sys.version_info[0]}_{args.backend}' + fext

 if args.verbose
 print(f'done!\nI Disparity output: {disparity_fname}', flush=True)
 if calcConf
 print(f'I Confidence output: {confidence_fname}', flush=True)

 # Save results to disk.
 try
 if args.output_mode == 0
 Image.fromarray(disparityColor).save(disparity_fname)
 if args.verbose
 print(f'I Output disparity image: {disparityColor.shape} '
 f'{disparityColor.dtype}', flush=True)
 elif args.output_mode == 1
 Image.fromarray(disparityU8).save(disparity_fname)
 if args.verbose
 print(f'I Output disparity image: {disparityU8.shape} '
 f'{disparityU8.dtype}', flush=True)
 elif args.output_mode == 2
 disparityS16.cpu().tofile(disparity_fname)
 if args.verbose
 print(f'I Output disparity image: {disparityS16.size} '
 f'{disparityS16.format}', flush=True)

 if calcConf
 if args.output_mode == 0 or args.output_mode == 1
 Image.fromarray(confidenceU8).save(confidence_fname)
 if args.verbose
 print(f'I Output confidence image: {confidenceU8.shape} '
 f'{confidenceU8.dtype}', flush=True)
 else
 confidenceU16.cpu().tofile(confidence_fname)
 if args.verbose
 print(f'I Output confidence image: {confidenceU16.size} '
 f'{confidenceU16.format}', flush=True)

 except
 raise ValueError(f'E Cannot write outputs: {disparity_fname}, {confidence_fname}\n'
 f'E Using output mode: {args.output_mode}')


if __name__ == '__main__'
 main()

#include <opencv2/core/version.hpp>
#if CV_MAJOR_VERSION >= 3
# include <opencv2/imgcodecs.hpp>
#else
# include <opencv2/contrib/contrib.hpp> // for colormap
# include <opencv2/highgui/highgui.hpp>
#endif

#include <opencv2/imgproc/imgproc.hpp>
#include <vpi/OpenCVInterop.hpp>

#include <vpi/Image.h>
#include <vpi/Status.h>
#include <vpi/Stream.h>
#include <vpi/algo/ConvertImageFormat.h>
#include <vpi/algo/Rescale.h>
#include <vpi/algo/StereoDisparity.h>

#include <cstring>
#include <iostream>
#include <sstream>

#define CHECK_STATUS(STMT) \
 do \
 { \
 VPIStatus status = (STMT); \
 if (status != VPI_SUCCESS) \
 { \
 char buffer[VPI_MAX_STATUS_MESSAGE_LENGTH]; \
 vpiGetLastStatusMessage(buffer, sizeof(buffer)); \
 std::ostringstream ss; \
 ss << "line " << __LINE__ << " " << vpiStatusGetName(status) << ": " << buffer; \
 throw std::runtime_error(ss.str()); \
 } \
 } while (0);

int main(int argc, char *argv[])
{
 // OpenCV 图像，将被 VPIImage 包裹。
 // 在此处定义它，以便在 wrapper 被销毁*之后*再销毁
 cv::Mat cvImageLeft, cvImageRight;

 // 将要使用的 VPI 对象
 VPIImage inLeft = NULL;
 VPIImage inRight = NULL;
 VPIImage tmpLeft = NULL;
 VPIImage tmpRight = NULL;
 VPIImage stereoLeft = NULL;
 VPIImage stereoRight = NULL;
 VPIImage disparity = NULL;
 VPIImage confidenceMap = NULL;
 VPIStream stream = NULL;
 VPIPayload stereo = NULL;

 int retval = 0;

 try
 {
 // =============================
 // 解析命令行参数

 if (argc != 4)
 {
 throw std::runtime_error(std::string("Usage: ") + argv[0] +
 " <cuda|ofa|ofa-pva-vic> <left image> <right image>");
 }

 std::string strBackend = argv[1];
 std::string strLeftFileName = argv[2];
 std::string strRightFileName = argv[3];

 uint64_t backends;

 if (strBackend == "cuda")
 {
 backends = VPI_BACKEND_CUDA;
 }
 else if (strBackend == "ofa")
 {
 backends = VPI_BACKEND_OFA;
 }
 else if (strBackend == "ofa-pva-vic")
 {
 backends = VPI_BACKEND_OFA | VPI_BACKEND_PVA | VPI_BACKEND_VIC;
 }
 else
 {
 throw std::runtime_error("Backend '" + strBackend +
 "' not recognized, it must be either cuda, ofa or ofa-pva-vic.");
 }

 // =====================
 // 加载输入图像
 cvImageLeft = cv::imread(strLeftFileName);
 if (cvImageLeft.empty())
 {
 throw std::runtime_error("Can't open '" + strLeftFileName + "'");
 }

 cvImageRight = cv::imread(strRightFileName);
 if (cvImageRight.empty())
 {
 throw std::runtime_error("Can't open '" + strRightFileName + "'");
 }

 // =================================
 // 分配所有需要的 VPI 资源

 int32_t inputWidth = cvImageLeft.cols;
 int32_t inputHeight = cvImageLeft.rows;

 // 创建将用于处理的流。
 CHECK_STATUS(vpiStreamCreate(0, &stream));

 // 现在我们将加载的图像包装到 VPIImage 对象中，以供 VPI 使用。
 // VPI 不会复制它，因此原始图像必须始终在作用域内。
 CHECK_STATUS(vpiImageCreateWrapperOpenCVMat(cvImageLeft, 0, &inLeft));
 CHECK_STATUS(vpiImageCreateWrapperOpenCVMat(cvImageRight, 0, &inRight));

 // 输入预处理所需的格式转换参数
 VPIConvertImageFormatParams convParams;
 CHECK_STATUS(vpiInitConvertImageFormatParams(&convParams));

 // 初始化默认参数
 VPIStereoDisparityEstimatorCreationParams createParams;
 CHECK_STATUS(vpiInitStereoDisparityEstimatorCreationParams(&createParams));

 // 选择最大视差，该视差适用于 chair_stereo_{left,right}_1920.png 文件
 createParams.maxDisparity = 256;

 // 输入立体图像对的默认格式和大小（某些后端需要调整，请参见下文）
 VPIImageFormat stereoFormat = VPI_IMAGE_FORMAT_Y8_ER;

 int stereoWidth = inputWidth;
 int stereoHeight = inputHeight;

 // 输出的默认格式和大小
 VPIImageFormat disparityFormat = VPI_IMAGE_FORMAT_S16;

 int outputWidth = inputWidth;
 int outputHeight = inputHeight;

 // 覆盖一些后端相关的参数
 if (strBackend.find("ofa") != std::string::npos)
 {
 // 使用 OFA 的实现需要 BL 输入
 stereoFormat = VPI_IMAGE_FORMAT_Y8_ER_BL;

 if (strBackend == "ofa")
 {
 // 当单独使用 OFA 时，输出也必须是 BL
 disparityFormat = VPI_IMAGE_FORMAT_S16_BL;
 }

 // 将下采样因子与 OFA 一起使用可提高性能
 createParams.downscaleFactor = 2;
 outputWidth = (inputWidth + createParams.downscaleFactor - 1) / createParams.downscaleFactor;
 outputHeight = (inputHeight + createParams.downscaleFactor - 1) / createParams.downscaleFactor;

 // 当使用 OFA+PVA+VIC 后端时，包括 downscaleFactor 在内的输出宽度必须至少为 max(64, maxDisparity/downscaleFactor)
 // OFA+PVA+VIC 后端被使用
 if (strBackend.find("pva") != std::string::npos)
 {
 int minWidth = std::max(createParams.maxDisparity / createParams.downscaleFactor, outputWidth);
 outputWidth = std::max(64, minWidth);
 outputHeight = (inputHeight * outputWidth) / inputWidth;
 stereoWidth = outputWidth * createParams.downscaleFactor;
 stereoHeight = outputHeight * createParams.downscaleFactor;
 }
 }

 // 为立体视差算法创建有效负载。
 // 有效负载在图像对象之前创建，以便可以使用错误捕获不支持的后端。
 CHECK_STATUS(vpiCreateStereoDisparityEstimator(backends, stereoWidth, stereoHeight, stereoFormat, &createParams,
 &stereo));

 // 创建将存储视差图的输出图像。
 CHECK_STATUS(vpiImageCreate(outputWidth, outputHeight, disparityFormat, 0, &disparity));

 // 创建输入立体图像
 CHECK_STATUS(vpiImageCreate(stereoWidth, stereoHeight, stereoFormat, 0, &stereoLeft));
 CHECK_STATUS(vpiImageCreate(stereoWidth, stereoHeight, stereoFormat, 0, &stereoRight));

 // 如果后端可以支持，则创建置信度图像
 if (strBackend == "ofa-pva-vic" || strBackend == "cuda")
 {
 CHECK_STATUS(vpiImageCreate(outputWidth, outputHeight, VPI_IMAGE_FORMAT_U16, 0, &confidenceMap));
 }

 // 如果需要重新缩放输入，则为初始格式转换创建临时图像。
 bool const isRescaleRequired = (stereoWidth != inputWidth) || (stereoHeight != inputHeight);
 if (isRescaleRequired)
 {
 CHECK_STATUS(vpiImageCreate(inputWidth, inputHeight, stereoFormat, 0, &tmpLeft));
 CHECK_STATUS(vpiImageCreate(inputWidth, inputHeight, stereoFormat, 0, &tmpRight));
 }

 // ================
 // 处理阶段

 // 从默认参数开始，并根据使用的后端覆盖某些值。
 VPIStereoDisparityEstimatorParams submitParams;
 CHECK_STATUS(vpiInitStereoDisparityEstimatorParams(&submitParams));
 if (strBackend == "ofa-pva-vic")
 {
 // INFERENCE 置信度类型在使用 OFA+PVA+VIC 后端时可获得更好的性能。唯一的权衡是
 // 基于深度学习的置信度图不容易表示为左右视差估计的函数，
 // 与 ABSOLUTE 或 RELATIVE 置信度类型相反。
 submitParams.confidenceType = VPI_STEREO_CONFIDENCE_INFERENCE;
 }
 else if (strBackend == "cuda")
 {
 // chair_stereo_{left,right}_1920.png 输入受益于 CUDA 的更高置信度阈值
 submitParams.confidenceThreshold = UINT16_MAX - 10000;
 }

 // -----------------
 // 预处理输入
 if (isRescaleRequired)
 {
 // 我们只需要使用 CUDA 进行转换，因为我们从 OpenCV 以默认的 BGR 格式加载了图像
 // 并且 VIC 后端不支持 3 通道 RGB/BGR 图像格式。
 // 或者，我们可以加载灰度图像并在 VIC 上一次操作中处理转换+重新缩放。

 // 使用 CUDA 将 opencv 输入转换为灰度格式
 CHECK_STATUS(vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, inLeft, tmpLeft, &convParams));
 CHECK_STATUS(vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, inRight, tmpRight, &convParams));

 // 在 VIC 上重新缩放
 CHECK_STATUS(
 vpiSubmitRescale(stream, VPI_BACKEND_VIC, tmpLeft, stereoLeft, VPI_INTERP_LINEAR, VPI_BORDER_CLAMP, 0));
 CHECK_STATUS(vpiSubmitRescale(stream, VPI_BACKEND_VIC, tmpRight, stereoRight, VPI_INTERP_LINEAR,
 VPI_BORDER_CLAMP, 0));
 }
 else
 {
 // 使用 CUDA 将 opencv 输入转换为灰度格式
 CHECK_STATUS(vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, inLeft, stereoLeft, &convParams));
 CHECK_STATUS(vpiSubmitConvertImageFormat(stream, VPI_BACKEND_CUDA, inRight, stereoRight, &convParams));
 }

 // ------------------------------
 // 进行立体视差估计

 // 使用输入和输出图像提交它
 CHECK_STATUS(vpiSubmitStereoDisparityEstimator(stream, backends, stereo, stereoLeft, stereoRight, disparity,
 confidenceMap, &submitParams));

 // 等待算法完成处理
 CHECK_STATUS(vpiStreamSync(stream));

 // ========================================
 // 输出预处理和保存到磁盘
 // 锁定输出以在 cpu 内存中检索其数据
 VPIImageData data;
 CHECK_STATUS(vpiImageLockData(disparity, VPI_LOCK_READ, VPI_IMAGE_BUFFER_HOST_PITCH_LINEAR, &data));

 // 从此图像创建一个 OpenCV 矩阵
 cv::Mat cvDisparity;
 CHECK_STATUS(vpiImageDataExportOpenCVMat(data, &cvDisparity));

 // 缩放结果并将其写入磁盘。视差以 Q10.5 格式表示，
 // 因此要将其映射到浮点数，需要除以 32。然后将生成的视差范围，
 // 从 0 到 maxDisparity 映射到 0-255 以获得正确的输出。
 cvDisparity.convertTo(cvDisparity, CV_8UC1, 255.0 / (32 * createParams.maxDisparity), 0);

 // 应用 JET 颜色映射以将视差转换为颜色。
 // 偏红色调表示更靠近相机的物体，偏蓝色调表示更远的物体。
 cv::Mat cvDisparityColor;
 applyColorMap(cvDisparity, cvDisparityColor, cv::COLORMAP_JET);

 // 输出处理完成，不要忘记解锁。
 CHECK_STATUS(vpiImageUnlock(disparity));

 // 如果我们有置信度图，也调整它以进行显示并将其写入磁盘。
 if (confidenceMap)
 {
 // 锁定图像数据并导出到 cv::Mat
 VPIImageData data;
 CHECK_STATUS(vpiImageLockData(confidenceMap, VPI_LOCK_READ, VPI_IMAGE_BUFFER_HOST_PITCH_LINEAR, &data));
 cv::Mat cvConfidence;
 CHECK_STATUS(vpiImageDataExportOpenCVMat(data, &cvConfidence));

 // 置信度图的范围从 0 到 65535，我们将其缩放到 [0-255]。
 cvConfidence.convertTo(cvConfidence, CV_8UC1, 255.0 / 65535, 0);
 imwrite("confidence_" + strBackend + ".png", cvConfidence);

 CHECK_STATUS(vpiImageUnlock(confidenceMap));

 // 当像素置信度为 0 时，我们希望视差图像中的颜色为黑色。
 cv::Mat cvMask;
 threshold(cvConfidence, cvMask, 1, 255, cv::THRESH_BINARY);
 cvtColor(cvMask, cvMask, cv::COLOR_GRAY2BGR);
 bitwise_and(cvDisparityColor, cvMask, cvDisparityColor);
 }

 imwrite("disparity_" + strBackend + ".png", cvDisparityColor);
 }
 catch (std::exception &e)
 {
 std::cerr << e.what() << std::endl;
 retval = 1;
 }

 // ========
 // 清理

 // 首先销毁流可确保提交给它的所有工作
 // 都已完成。
 vpiStreamDestroy(stream);

 // 只有这样我们才能销毁其他对象，因为我们确信它们
 // 不再被使用。

 vpiImageDestroy(inLeft);
 vpiImageDestroy(inRight);
 vpiImageDestroy(tmpLeft);
 vpiImageDestroy(tmpRight);
 vpiImageDestroy(stereoLeft);
 vpiImageDestroy(stereoRight);
 vpiImageDestroy(confidenceMap);
 vpiImageDestroy(disparity);
 vpiPayloadDestroy(stereo);

 return retval;
}