h264parse 损坏/无效 nal

Question

我正在尝试使用gst-launch-1.0 模拟发送和接收 mpeg 视频。

发送方管道： gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080,framerate=15/1 ! queue ! x264enc bitrate=4000 ! queue ! mpegtsmux ! rtpmp2tpay ! udpsink host=224.10.10.10 port=15004

接收器管道： gst-launch-1.0 -v rtpbin name=rtpbin udpsrc caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)MP2T,payload=(int)33,seqnum-offset=(uint)2803,timestamp-offset=(uint)2170591411, ssrc=(uint)2276926567" port=15004 multicast-group=224.10.10.10 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtpmp2tdepay ! tsdemux ! h264parse ! capsfilter caps=video/x-h264,alignment=au,stream-format=avc ! avdec_h264 ! fpsdisplaysink sync=1 udpsrc port=18889 ! rtpbin.recv_rtcp_sink_0

扣除： gst-launch-1.0 -v rtpbin name=rtpbin udpsrc caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)MP2T,payload=(int)33,seqnum-offset=(uint)2803,timestamp-offset=(uint)2170591411, ssrc=(uint)2276926567" port=15004 multicast-group=224.10.10.10 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtpmp2tdepay ! tsdemux ! decodebin ! fpsdisplaysink sync=1 udpsrc port=18889 ! rtpbin.recv_rtcp_sink_0

首先运行接收器，然后发送器立即按预期显示视频窗口。

但是当发送方首先启动时，接收方管道会挂起大约 10 秒，显示大量这些消息：

0:00:01.654820616 23285 0x56008c3bb640 WARN               h264parse gsth264parse.c:1349:gst_h264_parse_handle_frame:<h264parse0> broken/invalid nal Type: 1 Slice, Size: 32773 will be dropped

gst-launch-1.0 --gst-version 返回：

GStreamer Core Library version 1.16.2

Answer 1

我认为它类似于my question here。抛出此错误是因为您的 h264parse 元素尚未收到 SPS/PPS 数据。

抛出警告from here:

if (!gst_h264_parse_process_nal (h264parse, &nalu)) {
  GST_WARNING_OBJECT (h264parse,
      "broken/invalid nal Type: %d %s, Size: %u will be dropped",
      nalu.type, _nal_name (nalu.type), nalu.size);
  [...]
}

对于切片，这会失败 here:

    case GST_H264_NAL_SLICE:
    case GST_H264_NAL_SLICE_DPA:
    case GST_H264_NAL_SLICE_DPB:
    case GST_H264_NAL_SLICE_DPC:
    case GST_H264_NAL_SLICE_IDR:
    case GST_H264_NAL_SLICE_EXT:
      /* expected state: got-sps|got-pps (valid picture headers) */
      h264parse->state &= GST_H264_PARSE_STATE_VALID_PICTURE_HEADERS;
      if (!GST_H264_PARSE_STATE_VALID (h264parse,
              GST_H264_PARSE_STATE_VALID_PICTURE_HEADERS))
        return FALSE;

我认为这是因为当您首先启动发送方时，SPS/PPS 帧会在接收方启动之前发出，因此会丢失一段时间。您可以尝试让发件人更频繁地发出此数据。我认为例如 h264parse config-interval=-1 会在每个 IDR 帧中发送它。