DeepStream 中的 AV 同步#

DeepStream-6.0 及更高版本启用了 AV 同步支持。目前它处于 alpha 开发阶段。示例应用程序 deepstream-avsync-app 也已在 app/sample_apps/deepstream-avsync 中提供以供参考。本文档提供了示例 gstreamer 管道。

用于测试的 RTMP/RTSP 输入流设置#

RTMP 服务器设置#

$ apt-get update
$ apt-get install nginx -y
$ apt-get install libnginx-mod-rtmp -y
$ vim /etc/nginx/nginx.conf #add below content
  rtmp {
    server {
    listen 1935;
    chunk_size 4096;
    application live {
    live on;
    record off;
    }
   }
  }
$ service nginx restart

使用 ffmpeg 模拟 2 个 RTMP 流的命令#

$ sudo ffmpeg -re -i <file_name1.mp4> -vcodec copy -1 -c:a aac -b:a 160k -ar 48000 -strict -2 -f flv rtmp://<host ip address:port>/live/test1
$ sudo ffmpeg -re -i <file_name2.mp4> -vcodec copy -1 -c:a aac -b:a 160k -ar 48000 -strict -2 -f flv rtmp://<host ip address:port>/live/test2

RTSP 服务器设置#

$ cvlc <file_name.mp4> :sout=#gather:rtp{sdp=rtsp://:<port>/file-stream} :network-caching=1500 :sout-all :sout-keep

可以使用 URL：rtsp://<host_ip_address:port>/file-stream 访问从上述命令生成的 rtsp 流。

AVSync 参考管道#

注意

  1. RTMP src 不是实时源。您需要添加 fakesrc 并设置 is-live=true，并将其连接到 audiomixer。这确保即使其中一个来源的数据不可用，它也不会中断。

  2. 在 nvstreammux 上为 rtmp 输入源设置了 250 毫秒的最大延迟。设置该值是因为从 rtmpsrc 到 nvstreammux 的缓冲区所需的最大延迟为 250 毫秒。您可以根据您的要求调整此值。出于相同的原因，音频路径上的 audiomixer 也设置了相同的值。

  3. RTSP 输出将在 rtsp://<host_ip_address:8554>/ds-test 上可用。

  4. 如果使用 VLC 播放器播放文件输出，您可能会遇到 AV 同步不同步的情况。在这种情况下，请尝试 ffmpeg 或 gst-launch 管道。这可能是 VLC 播放器特有的问题。

  5. 确保音频流具有相同的编码配置。（例如，采样率）。如果这些配置不同，audiomixer 将会崩溃。您可以在 audiomixer 之前添加 audioconvert 和 audioresample 以针对给定源，以便 audiomixer 的所有输入都具有相同的格式和采样率。

  6. YouTube™ 直播上的 RTMP 输出需要一段时间才能出现（约 30-40 秒）。这可能是因为 YouTube™ 播放器的缓冲机制。

  7. 在 RTMP 输出的情况下，在运行管道之前，请确保 YouTube™ 直播页面已刷新并准备好接受传入数据。

  8. 如果发生数据包丢失，rtsp 输出可能看起来已损坏。在这种情况下，请降低比特率并重试。

具有现有 nvstreammux 组件的管道#

RTMP_IN->RTMP_OUT#

input1=rtmp://<host ip address:port>/live/test1

input2=rtmp://<host ip address:port>/live/test2

output=<RTMP url>

 e.g.
 rtmp://a.rtmp.youtube.com/live2/<key> For Youtube live
 rtmp://<host ip address:port>/live/test For Host

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 max-latency=250000000 batched-push-timeout=33333 width=1920 height=1080 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true !  rtmpsink location=$output async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer  latency=250000000 ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer. -e

FILE_IN->RTSP_OUT#

input1=file:///AV_I_frames_1.mp4

input2=file:///AV_I_frames_2.mp4

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 batched-push-timeout=33333 width=   1920 height=1080 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvrtspoutsinkbin name=r uridecodebin3 uri=$input2 name=demux2  ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! r.  demux2. ! queue ! audioconvert ! mixer. fakesrc num-buffers=0 is-live=1 ! mixer. -e

FILE_IN->RTMP_OUT#

input1=file:///AV_I_frames_1.mp4

input2=file:///AV_I_frames_2.mp4

output=<RTMP url>

e.g.

rtmp://a.rtmp.youtube.com/live2/<key> For Youtube live

rtmp://<host ip address:port>/live/test For Host

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 batched-push-timeout=33333 width=1920 height=1080 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc  ! h264parse ! queue ! flvmux name=mux streamable=true !  rtmpsink location=$output async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer.   fakesrc num-buffers=0 is-live=1 ! mixer. -e

RTMP_IN->FILE_OUT#

input1=rtmp://<host ip address:port>/live/test1

input2=rtmp://<host ip address:port>/live/test2

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 batched-push-timeout=33333 width=1920 height=1080 sync-inputs=1 max-latency=250000000 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! filesink location=out.flv  async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer latency=250000000 name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer. -e

RTSP_IN->FILE_OUT#

input1=<rtsp url>

input2=<rtsp url>

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 batched-push-timeout=33333 width=1920 height=1080 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! filesink location=out.flv  async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer. -e

FILE_IN->FILE_OUT#

input1=file:///AV_I_frames_1.mp4

input2=file:///AV_I_frames_2.mp4

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 batched-push-timeout=33333 width=1920 height=1080 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! filesink location=out.flv  async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer. fakesrc num-buffers=0 is-live=1 ! mixer. -e

RTSP_IN->RTSP_OUT#

input1=<rtsp url>

input2=<rtsp url>

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 batched-push-timeout=33333 width=1920 height=1080 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvrtspoutsinkbin name=r uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! r.  demux2. ! queue ! audioconvert ! mixer. -e

RTMP_IN->RTSP_OUT#

input1=rtmp://<host ip address:port>/live/test1

input2=rtmp://<host ip address:port>/live/test2

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 batched-push-timeout=33333 width=1920 height=1080 sync-inputs=1 max-latency=250000000 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvrtspoutsinkbin name=r uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer latency=250000000 name=mixer ! queue ! r.  demux2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer. -e

具有现有 nvstreammux 的参考 AVSync + ASR（自动语音识别）管道#

注意

这些管道演示了如何在 avsync 管道中添加 ASR 插件。

要可视化视频帧上 ASR 的叠加文本输出，请参考 deepstream-avsync-app 应用程序。

RTMP_IN->RTMP_OUT#

input1=rtmp://<host ip address:port>/live/test1
input2=rtmp://<host ip address:port>/live/test2

output=<RTMP url>
e.g.:
rtmp://a.rtmp.youtube.com/live2/<key> #For YouTube™ live
rtmp://<host ip address:port>/live/test #For Host

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux max-latency=250000000 batched-push-timeout=33333 width=1920 height=1080 batch-size=2 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvdsosd ! queue ! nvvideoconvert ! queue ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! rtmpsink location=$output async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1                                         demux1. ! queue ! tee name=t1 t1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer latency=250000000 ! queue ! avenc_aac ! aacparse ! queue ! mux.                   t1. ! queue ! audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1" ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx"  ! fakesink sync=0 async=0                demux2. ! queue ! tee name=t2 t2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer.                t2. ! queue ! audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1"  ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx"  ! fakesink sync=0 async=0 -e

注意

确保 ASR 服务正在运行（有关详细信息，请参阅 deepstream-avsync-app 的 README）。为确保 gRPC 库可访问，请使用 $source ~/.profile 设置 LD _LIBRARY_PATH。在运行上述命令之前，请更新 riva_asr_grpc_jasper_conf.yml 的正确路径。

RTSP_IN->RTMP_OUT#

input1=<rtsp url>
input2=<rtsp_url>

output=<RTMP url>
e.g.:
rtmp://a.rtmp.youtube.com/live2/<key> #For Youtube live
rtmp://<host ip address:port>/live/test #For Host

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batched-push-timeout=33333 width=1920 height=1080 batch-size=2 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=flvmux streamable=true !  rtmpsink location=$output async=0 qos=0 sync=1                                        uridecodebin3 uri= $input2 name =demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1                     demux1. ! queue ! tee name=t1 t1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! flvmux.   t1. ! queue !  audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1" ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx" ! fakesink sync=0 async=0 qos=0                                 demux2. ! queue ! tee name=t2   t2. ! queue !  audioconvert ! mixer.                    t2. ! queue ! audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1" ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx" ! fakesink sync=0 async=0 qos=0 -e

注意

确保 ASR 服务正在运行（有关详细信息，请参阅 deepstream-avsync-app 的 README）。为确保 gRPC 库可访问，请使用 $source ~/.profile 设置 LD _LIBRARY_PATH。在运行上述命令之前，请更新 riva_asr_grpc_jasper_conf.yml 的正确路径。

FILE_IN->RTMP_OUT#

input1=file:///AV_I_frames_1.mp4
input2=file:///AV_I_frames_2.mp4

output=<RTMP url>
e.g.:
rtmp://a.rtmp.youtube.com/live2/<key> #For Youtube live
rtmp://<host ip address:port>/live/test #For Host

gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 batched-push-timeout=33333 width=1920 height=1080 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvdsosd ! queue ! nvvideoconvert ! queue ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! rtmpsink location=$output async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1                         demux1. ! queue ! tee name=t1 t1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux.       t1. ! queue ! audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1" ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx"  ! fakesink sync=0 async=0              demux2. ! queue ! tee name=t2 t2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer.               t2. ! queue ! audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1"  ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx"  ! fakesink sync=0 async=0 -e

注意

确保 ASR 服务正在运行（有关详细信息，请参阅 deepstream-avsync-app 的 README）。为确保 gRPC 库可访问，请使用 $source ~/.profile 设置 LD _LIBRARY_PATH。在运行上述命令之前，请更新 riva_asr_grpc_jasper_conf.yml 的正确路径。

具有新的 nvstreammux 组件的管道#

您可以通过导出 USE_NEW_NVSTREAMMUX=yes 来启用新的 nvstreammux（Beta 质量）。有关更多信息，请参阅 NVIDIA DeepStream SDK 开发者指南 6.1 版本 中的 :doc: “Gst-nvstreammux New (Beta)” 部分。

注意

现有的 nvstreammux 功能将在未来弃用。

RTMP_IN->RTMP_OUT#

input1=rtmp://<host ip address:port>/live/test1

input2=rtmp://<host ip address:port>/live/test2

output=<RTMP url>
e.g.
rtmp://a.rtmp.youtube.com/live2/<key> For Youtube live

rtmp://<host ip address:port>/live/test For Host

USE_NEW_NVSTREAMMUX=yes  gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 max-latency=250000000 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true !  rtmpsink location=$output async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer  latency=250000000 ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer. -e

RTSP_IN->RTMP_OUT#

input1=<rtsp url>
input2=<rtsp url>

output=<RTMP url>
e.g.
rtmp://a.rtmp.youtube.com/live2/<key> For Youtube live
rtmp://<host ip address:port>/live/test For Host

USE_NEW_NVSTREAMMUX=yes  gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc  ! h264parse ! queue ! flvmux name=mux streamable=true !  rtmpsink location=$output async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer. -e

FILE_IN->RTSP_OUT#

input1=file:///AV_I_frames_1.mp4

input2=file:///AV_I_frames_2.mp4

USE_NEW_NVSTREAMMUX=yes  gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2  sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvrtspoutsinkbin name=r uridecodebin3 uri=$input2 name=demux2  ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! r.  demux2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer. -e

FILE_IN->RTMP_OUT#

input1=file:///AV_I_frames_1.mp4

input2=file:///AV_I_frames_2.mp4

output=<RTMP url>

 e.g.
 rtmp://a.rtmp.youtube.com/live2/<key> For Youtube live
 rtmp://<host ip address:port>/live/test For Host

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc  ! h264parse ! queue ! flvmux name=mux streamable=true !  rtmpsink location=$output async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer. fakesrc num-buffers=0 is-live=1 ! mixer. -e

RTMP_IN->FILE_OUT#

input1=rtmp://<host ip address:port>/live/test1

input2=rtmp://<host ip address:port>/live/test2

USE_NEW_NVSTREAMMUX=yes  gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 sync-inputs=1 max-latency=250000000 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! filesink location=out.flv  async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer latency=250000000 name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer. -e

RTSP_IN->FILE_OUT#

input1=<rtsp url>

input2=<rtsp url>

USE_NEW_NVSTREAMMUX=yes  gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! filesink location=out.flv  async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer. -e

FILE_IN->FILE_OUT#

input1=file:///AV_I_frames_1.mp4

input2=file:///AV_I_frames_2.mp4

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2  sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! filesink location=out.flv  async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux. demux2. ! queue ! audioconvert ! mixer. fakesrc num-buffers=0 is-live=1 ! mixer. -e

RTSP_IN->RTSP_OUT#

input1=<rtsp url>

input2=<rtsp url>

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2  sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvrtspoutsinkbin name=r uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! r.  demux2. ! queue ! audioconvert ! mixer. -e

RTMP_IN->RTSP_OUT#

input1=rtmp://<host ip address:port>/live/test1

input2=rtmp://<host ip address:port>/live/test2

USE_NEW_NVSTREAMMUX=yes  gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 sync-inputs=1 max-latency=250000000 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvrtspoutsinkbin name=r uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1 demux1. ! queue ! audioconvert ! mixer.sink_0 audiomixer latency=250000000 name=mixer ! queue ! r.  demux2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer. -e

参考 AVSync + ASR 管道（使用新的 nvstreammux）#

RTMP_IN->RTMP_OUT#

input1=rtmp://<host ip address:port>/live/test1
input2=rtmp://<host ip address:port>/live/test2

output=<RTMP url>
e.g.:
rtmp://a.rtmp.youtube.com/live2/<key> #For Youtube live
rtmp://<host ip address:port>/live/test #For Host

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux max-latency=250000000 batch-size=2 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvdsosd ! queue ! nvvideoconvert ! queue ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! rtmpsink location=$output async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1                            demux1. ! queue ! tee name=t1 t1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer latency=250000000 ! queue ! avenc_aac ! aacparse ! queue ! mux.            t1. ! queue ! audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1" ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx"  ! fakesink sync=0 async=0      demux2. ! queue ! tee name=t2 t2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer.             t2. ! queue ! audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1"  ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx"  ! fakesink sync=0 async=0 -e

注意

确保 ASR 服务正在运行（有关详细信息，请参阅 deepstream-avsync-app 的 README）。为确保 gRPC 库可访问，请使用 $source ~/.profile 设置 LD _LIBRARY_PATH。在运行上述命令之前，请更新 riva_asr_grpc_jasper_conf.yml 的正确路径。

RTSP_IN->RTMP_OUT#

input1=<rtsp url>
input2=<rtsp_url>

output=<RTMP url>
e.g.:
rtmp://a.rtmp.youtube.com/live2/<key> #For Youtube live
rtmp://<host ip address:port>/live/test #For Host

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=flvmux streamable=true !  rtmpsink location=$output async=0 qos=0 sync=1                            uridecodebin3 uri= $input2 name =demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1                   demux1. ! queue ! tee name=t1 t1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! flvmux.   t1. ! queue !  audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1" ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx" ! fakesink sync=0 async=0 qos=0               demux2. ! queue ! tee name=t2   t2. ! queue !  audioconvert ! mixer.                t2. ! queue ! audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1" ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx" ! fakesink sync=0 async=0 qos=0 -e

注意

确保 ASR 服务正在运行（有关详细信息，请参阅 deepstream-avsync-app 的 README）。为确保 gRPC 库可访问，请使用 $source ~/.profile 设置 LD _LIBRARY_PATH。在运行上述命令之前，请更新 riva_asr_grpc_jasper_conf.yml 的正确路径。

FILE_IN->RTMP_OUT#

input1=file:///AV_I_frames_1.mp4
input2=file:///AV_I_frames_2.mp4

output=<RTMP url>
e.g.:
rtmp://a.rtmp.youtube.com/live2/<key> #For Youtube live
rtmp://<host ip address:port>/live/test #For Host

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=2 sync-inputs=1 name=mux1 ! queue ! nvmultistreamtiler width=480 height=360 ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvdsosd ! queue ! nvvideoconvert ! queue ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! rtmpsink location=$output async=0 qos=0 sync=1 uridecodebin3 uri=$input2 name=demux2 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_1                         demux1. ! queue ! tee name=t1 t1. ! queue ! audioconvert ! mixer.sink_0 audiomixer name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux.       t1. ! queue ! audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1" ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx"  ! fakesink sync=0 async=0                demux2. ! queue ! tee name=t2 t2. ! queue ! audioconvert ! mixer.  fakesrc num-buffers=0 is-live=1 ! mixer.             t2. ! queue ! audioconvert ! "audio/x-raw, format=(string)S16LE, channels=(int)1"  ! audioresample ! "audio/x-raw, rate=(int)16000" ! nvdsasr config-file= riva_asr_grpc_jasper_conf.yml customlib-name="libnvds_riva_asr_grpc.so" create-speech-ctx-func="create_riva_asr_grpc_ctx"  ! fakesink sync=0 async=0 -e

注意

确保 ASR 服务正在运行（有关详细信息，请参阅 deepstream-avsync-app 的 README）。为确保 gRPC 库可访问，请使用 $source ~/.profile 设置 LD _LIBRARY_PATH。在运行上述命令之前，请更新 riva_asr_grpc_jasper_conf.yml 的正确路径。

带有 audiomuxer 的 Gst-pipeline（单源，不带 ASR + 新的 nvstreammux）#

RTMP_IN->FILE_OUT#

input1=rtmp://<host ip address:port>/live/test1

USE_NEW_NVSTREAMMUX=yes  gst-launch-1.0 uridecodebin3 uri=$input1 name=demux1 ! queue ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! mux1.sink_0 nvstreammux batch-size=1 sync-inputs=1 max-latency=250000000 name=mux1 ! queue ! nvmultistreamtiler ! nvvideoconvert ! "video/x-raw(memory:NVMM)" ! nvv4l2h264enc ! h264parse ! queue ! flvmux name=mux streamable=true ! filesink location=out.flv  async=0 qos=0 sync=1 demux1. ! queue ! audioconvert ! audiomux.sink_0 nvstreammux name=audiomux batch-size=1 max-latency=250000000 sync-inputs=1 ! nvstreamdemux name=audiodemux audiodemux.src_0  ! audioconvert ! mixer.sink_0 audiomixer latency=250000000 name=mixer ! queue ! avenc_aac ! aacparse ! queue ! mux. fakesrc num-buffers=0 is-live=1 ! mixer. -e
在本页中