【发布时间】：2016-07-19 20:25:28
【问题描述】：
问题：
在我的例子中，来自 h264 编码器的比特流由实时摄像头提供。比特流由 live555 库使用 RTSP 协议发送。 RTSP 数据包由 rtsp 客户端之一接收，例如 vlc 播放器（最重要的）、totem 播放器、mplayer。 我需要知道如何设置较低的帧率，因为我的相机产生 9 fps，但大多数 RTSP 客户端都期望默认为 25 fps，这会导致视频流无法正常播放（缓冲、丢弃延迟数据包等）。
问题：
我想知道 RTSP 客户端（播放软件）如何从比特流中读取（或评估）帧速率。这些信息存储在哪里以及如何发送到播放软件？
更多信息：
我正在使用来自 live555 的 testOnDemandRTSPServer 示例代码。我已经从 live555 lib 编写了 DeviceSource 类的实现。我已正确设置演示时间：
gettimeofday(&fPresentationTime, NULL);
fDurationInMicroseconds = 1100000;
Sequence Parameter Set NAL 还包含正确的时序信息：
timing_info_present_flag : 1
num_units_in_tick : 1001
time_scale : 9000
fixed_frame_rate_flag : 1
RTSP Header 包含帧率：
a=framerate:9.0
不幸的是，它没有帮助。我认为视频播放器会平静地等待新帧，并且帧率将与接收另一个 RTP 数据包（另一个 NAL）的频率相关。
视频播放器的行为方式：
VLC 播放器 - 视频断断续续，因为 vlc 需要更早的下一帧（25 fps 而不是 9 fps）。结果，vlc 将缓冲时间增加到例如10 秒，在此期间视频被冻结。
图腾播放器 - 来自 SPS Nal 的计时信息由图腾解析。帧率显示正常，但视频仍然不稳定（延迟约 500 毫秒）。似乎正在等待更多帧（每秒 25 帧）。
Mplayer 识别错误的帧率值，显示 25 fps，但视频非常流畅且质量很好：
mplayer ffmpeg://rtsp://192.168.1.82:8554/testStream
.
.
.
VIDEO: [H264] 320x240 0bpp 25.000 fps 0.0 kbps ( 0.0 kbyte/s)
Live555 RTSP 客户端测试应用 ./live555/testProgs/testRTSPClient rtsp://192.168.1.82:8554/testStream
Opening connection to 192.168.1.82, port 8554...
...remote connection opened
Sending request: DESCRIBE rtsp://192.168.1.82:8554/testStream RTSP/1.0
CSeq: 2
User-Agent: ./testRTSPClient (LIVE555 Streaming Media v2016.02.22)
Accept: application/sdp
Received 675 new bytes of response data.
Received a complete DESCRIBE response:
RTSP/1.0 200 OK
CSeq: 2
Date: Fri, Jan 01 1988 00:00:23 GMT
Content-Base: rtsp://192.168.1.82:8554/testStream/
Content-Type: application/sdp
Content-Length: 506
v=0
o=- 567993603275971 1 IN IP4 192.168.1.82
s=Session streamed by "testOnDemandRTSPServer"
i=testStream
t=0 0
a=tool:LIVE555 Streaming Media v2016.02.22
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:Session streamed by "testOnDemandRTSPServer"
a=x-qt-text-inf:testStream
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:500
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=42801E;sprop-parameter-sets=Z0KAHpWgUHxA,aM44gA==
a=framerate:9.0
a=control:track1
[URL:"rtsp://192.168.1.82:8554/testStream/"]: Got a SDP description:
v=0
o=- 567993603275971 1 IN IP4 192.168.1.82
s=Session streamed by "testOnDemandRTSPServer"
i=testStream
t=0 0
a=tool:LIVE555 Streaming Media v2016.02.22
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:Session streamed by "testOnDemandRTSPServer"
a=x-qt-text-inf:testStream
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:500
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=42801E;sprop-parameter-sets=Z0KAHpWgUHxA,aM44gA==
a=framerate:9.0
a=control:track1
[URL:"rtsp://192.168.1.82:8554/testStream/"]: Initiated the "video/H264" subsession (client ports 57660-57661)
Sending request: SETUP rtsp://192.168.1.82:8554/testStream/track1 RTSP/1.0
CSeq: 3
User-Agent: ./testRTSPClient (LIVE555 Streaming Media v2016.02.22)
Transport: RTP/AVP;unicast;client_port=57660-57661
Received 215 new bytes of response data.
Received a complete SETUP response:
RTSP/1.0 200 OK
CSeq: 3
Date: Fri, Jan 01 1988 00:00:23 GMT
Transport: RTP/AVP;unicast;destination=192.168.1.109;source=192.168.1.82;client_port=57660-57661;server_port=6970-6971
Session: 06B1DFA4;timeout=65
[URL:"rtsp://192.168.1.82:8554/testStream/"]: Set up the "video/H264" subsession (client ports 57660-57661)
[URL:"rtsp://192.168.1.82:8554/testStream/"]: Created a data sink for the "video/H264" subsession
Sending request: PLAY rtsp://192.168.1.82:8554/testStream/ RTSP/1.0
CSeq: 4
User-Agent: ./testRTSPClient (LIVE555 Streaming Media v2016.02.22)
Session: 06B1DFA4
Range: npt=0.000-
Received 96 new bytes of response data.
Received 94 new bytes of response data.
Received a complete PLAY response:
RTSP/1.0 200 OK
CSeq: 4
Date: Fri, Jan 01 1988 00:00:23 GMT
Range: npt=0.000-
Session: 06B1DFA4
RTP-Info: url=rtsp://192.168.1.82:8554/testStream/track1;seq=52833;rtptime=3161001994
[URL:"rtsp://192.168.1.82:8554/testStream/"]: Started playing session...
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 10874 bytes. Presentation time: 567993623.442288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 12054 bytes. Presentation time: 567993623.482288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 12227 bytes. Presentation time: 567993623.522288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 12281 bytes. Presentation time: 567993623.562288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 12384 bytes. Presentation time: 567993623.602288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 12129 bytes. Presentation time: 567993623.642288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 12141 bytes. Presentation time: 567993623.682288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 12287 bytes. Presentation time: 567993623.722288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 16415 bytes. Presentation time: 567993623.762288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 12360 bytes. Presentation time: 567993623.802288
Stream "rtsp://192.168.1.82:8554/testStream/"; video/H264: Received 12167 bytes. Presentation time: 567993623.842288
我通过工具 h264_analyze 解析了我的 NAL，下面是输出（只有少数第一个 NAL）：
!! Found NAL at offset 4 (0x0004), size 18 (0x0012)
XX 00 00 00 01 67 42 80 1E 95 A0 50 7C 84 00 00 0F
==================== NAL ====================
forbidden_zero_bit : 0
nal_ref_idc : 3
nal_unit_type : 7 ( Sequence parameter set )
======= SPS =======
profile_idc : 66
constraint_set0_flag : 1
constraint_set1_flag : 0
constraint_set2_flag : 0
constraint_set3_flag : 0
constraint_set4_flag : 0
constraint_set5_flag : 0
reserved_zero_2bits : 0
level_idc : 30
seq_parameter_set_id : 0
chroma_format_idc : 1
residual_colour_transform_flag : 0
bit_depth_luma_minus8 : 0
bit_depth_chroma_minus8 : 0
qpprime_y_zero_transform_bypass_flag : 0
seq_scaling_matrix_present_flag : 0
log2_max_frame_num_minus4 : 4
pic_order_cnt_type : 2
log2_max_pic_order_cnt_lsb_minus4 : 0
delta_pic_order_always_zero_flag : 0
offset_for_non_ref_pic : 0
offset_for_top_to_bottom_field : 0
num_ref_frames_in_pic_order_cnt_cycle : 0
num_ref_frames : 1
gaps_in_frame_num_value_allowed_flag : 0
pic_width_in_mbs_minus1 : 19
pic_height_in_map_units_minus1 : 14
frame_mbs_only_flag : 1
mb_adaptive_frame_field_flag : 0
direct_8x8_inference_flag : 0
frame_cropping_flag : 0
frame_crop_left_offset : 0
frame_crop_right_offset : 0
frame_crop_top_offset : 0
frame_crop_bottom_offset : 0
vui_parameters_present_flag : 1
=== VUI ===
aspect_ratio_info_present_flag : 0
aspect_ratio_idc : 0
sar_width : 0
sar_height : 0
overscan_info_present_flag : 0
overscan_appropriate_flag : 0
video_signal_type_present_flag : 0
video_format : 0
video_full_range_flag : 0
colour_description_present_flag : 0
colour_primaries : 0
transfer_characteristics : 0
matrix_coefficients : 0
chroma_loc_info_present_flag : 0
chroma_sample_loc_type_top_field : 0
chroma_sample_loc_type_bottom_field : 0
timing_info_present_flag : 1
num_units_in_tick : 1001
time_scale : 9000
fixed_frame_rate_flag : 1
nal_hrd_parameters_present_flag : 0
vcl_hrd_parameters_present_flag : 0
low_delay_hrd_flag : 0
pic_struct_present_flag : 0
bitstream_restriction_flag : 0
motion_vectors_over_pic_boundaries_flag : 0
max_bytes_per_pic_denom : 0
max_bits_per_mb_denom : 0
log2_max_mv_length_horizontal : 0
log2_max_mv_length_vertical : 0
num_reorder_frames : 0
max_dec_frame_buffering : 0
=== HRD ===
cpb_cnt_minus1 : 0
bit_rate_scale : 0
cpb_size_scale : 0
bit_rate_value_minus1[0] : 0
cpb_size_value_minus1[0] : 0
cbr_flag[0] : 0
initial_cpb_removal_delay_length_minus1 : 0
cpb_removal_delay_length_minus1 : 0
dpb_output_delay_length_minus1 : 0
time_offset_length : 0
!! Found NAL at offset 26 (0x001A), size 4 (0x0004)
XX 00 00 00 01 68 CE 38 80
==================== NAL ====================
forbidden_zero_bit : 0
nal_ref_idc : 3
nal_unit_type : 8 ( Picture parameter set )
======= PPS =======
pic_parameter_set_id : 0
seq_parameter_set_id : 0
entropy_coding_mode_flag : 0
pic_order_present_flag : 0
num_slice_groups_minus1 : 0
slice_group_map_type : 0
num_ref_idx_l0_active_minus1 : 0
num_ref_idx_l1_active_minus1 : 0
weighted_pred_flag : 0
weighted_bipred_idc : 0
pic_init_qp_minus26 : 0
pic_init_qs_minus26 : 0
chroma_qp_index_offset : 0
deblocking_filter_control_present_flag : 0
constrained_intra_pred_flag : 0
redundant_pic_cnt_present_flag : 0
transform_8x8_mode_flag : 0
pic_scaling_matrix_present_flag : 0
second_chroma_qp_index_offset : 0
!! Found NAL at offset 33 (0x0021), size 339 (0x0153)
XX 80 00 00 01 65 B8 04 04 3F FF F8 7A 28 03 EF BE
==================== NAL ====================
forbidden_zero_bit : 0
nal_ref_idc : 3
nal_unit_type : 5 ( Coded slice of an IDR picture )
======= Slice Header =======
first_mb_in_slice : 0
slice_type : 2 ( I slice )
pic_parameter_set_id : 0
frame_num : 0
field_pic_flag : 0
bottom_field_flag : 0
idr_pic_id : 0
pic_order_cnt_lsb : 0
delta_pic_order_cnt_bottom : 0
redundant_pic_cnt : 0
direct_spatial_mv_pred_flag : 0
num_ref_idx_active_override_flag : 0
num_ref_idx_l0_active_minus1 : 0
num_ref_idx_l1_active_minus1 : 0
cabac_init_idc : 0
slice_qp_delta : -16
sp_for_switch_flag : 0
slice_qs_delta : 0
disable_deblocking_filter_idc : 0
slice_alpha_c0_offset_div2 : 0
slice_beta_offset_div2 : 0
slice_group_change_cycle : 0
=== Prediction Weight Table ===
luma_log2_weight_denom : 0
chroma_log2_weight_denom : 0
=== Ref Pic List Reordering ===
ref_pic_list_reordering_flag_l0 : 0
ref_pic_list_reordering_flag_l1 : 0
=== Decoded Ref Pic Marking ===
no_output_of_prior_pics_flag : 0
long_term_reference_flag : 0
adaptive_ref_pic_marking_mode_flag : 0
!! Found NAL at offset 376 (0x0178), size 1897 (0x0769)
XX 00 00 00 01 41 E0 20 3F 36 5F FF 87 A2 80 03 7E
==================== NAL ====================
forbidden_zero_bit : 0
nal_ref_idc : 2
nal_unit_type : 1 ( Coded slice of a non-IDR picture )
======= Slice Header =======
first_mb_in_slice : 0
slice_type : 0 ( P slice )
pic_parameter_set_id : 0
frame_num : 1
field_pic_flag : 0
bottom_field_flag : 0
idr_pic_id : 0
pic_order_cnt_lsb : 0
delta_pic_order_cnt_bottom : 0
redundant_pic_cnt : 0
direct_spatial_mv_pred_flag : 0
num_ref_idx_active_override_flag : 0
num_ref_idx_l0_active_minus1 : 0
num_ref_idx_l1_active_minus1 : 0
cabac_init_idc : 0
slice_qp_delta : -15
sp_for_switch_flag : 0
slice_qs_delta : 0
disable_deblocking_filter_idc : 0
slice_alpha_c0_offset_div2 : 0
slice_beta_offset_div2 : 0
slice_group_change_cycle : 0
=== Prediction Weight Table ===
luma_log2_weight_denom : 0
chroma_log2_weight_denom : 0
=== Ref Pic List Reordering ===
ref_pic_list_reordering_flag_l0 : 0
ref_pic_list_reordering_flag_l1 : 0
=== Decoded Ref Pic Marking ===
no_output_of_prior_pics_flag : 0
long_term_reference_flag : 0
adaptive_ref_pic_marking_mode_flag : 0
!! Found NAL at offset 2277 (0x08E5), size 2989 (0x0BAD)
XX 00 00 00 01 41 E0 40 3F 37 0F 0C 3E D8 A0 00 4E
==================== NAL ====================
forbidden_zero_bit : 0
nal_ref_idc : 2
nal_unit_type : 1 ( Coded slice of a non-IDR picture )
======= Slice Header =======
first_mb_in_slice : 0
slice_type : 0 ( P slice )
pic_parameter_set_id : 0
frame_num : 2
field_pic_flag : 0
bottom_field_flag : 0
idr_pic_id : 0
pic_order_cnt_lsb : 0
delta_pic_order_cnt_bottom : 0
redundant_pic_cnt : 0
direct_spatial_mv_pred_flag : 0
num_ref_idx_active_override_flag : 0
num_ref_idx_l0_active_minus1 : 0
num_ref_idx_l1_active_minus1 : 0
cabac_init_idc : 0
slice_qp_delta : -15
sp_for_switch_flag : 0
slice_qs_delta : 0
disable_deblocking_filter_idc : 0
slice_alpha_c0_offset_div2 : 0
slice_beta_offset_div2 : 0
slice_group_change_cycle : 0
=== Prediction Weight Table ===
luma_log2_weight_denom : 0
chroma_log2_weight_denom : 0
=== Ref Pic List Reordering ===
ref_pic_list_reordering_flag_l0 : 0
ref_pic_list_reordering_flag_l1 : 0
=== Decoded Ref Pic Marking ===
no_output_of_prior_pics_flag : 0
long_term_reference_flag : 0
adaptive_ref_pic_marking_mode_flag : 0
!! Found NAL at offset 5270 (0x1496), size 4074 (0x0FEA)
XX 00 00 00 01 41 E0 60 10 CD B7 C1 BF B8 36 00 6A
【问题讨论】：
-
AFAIR fDurationInMicroseconds 对于实时源应设置为 0。不知道这是否会影响结果。
标签： video-streaming vlc rtsp frame-rate live555