【发布时间】：2018-12-05 17:44:50
【问题描述】：
MediaMuxer 已经把我逼疯了两天两夜 :-(
情况：我通过 UDP 接收到 H264 编码的 1280x720 视频流。 h264 流包含 NALU 1 - 切片和 NALU 5 - 关键帧（5 总是在 NALU 7 - SPS 和 NALU 8 - PPS 之前）。 这个流似乎是稳定的 30fps，每秒至少有一个 NALU 5 关键帧。比特率是可变的，但小于 4Mbps。 MediaCodec 成功解码流并将其呈现在表面视图中，以便该部分正常工作。
但现在我需要将 H.264 保存到本地 mpg 文件中。 我使用我拥有的所有 MediaFormat 信息设置了一个 MediaMuxer，并将流中的示例数据提供给它。 每个样本包含一个帧（NALU 1 或 5），发送到 MediaMuxer 的第一个数据是一个关键帧（NALU 5）。演示时间是根据帧数和帧率计算的。 所有涉及的方法都是从同一个线程调用的。
但从未创建 mpg 文件。正如您在下面的输出中看到的那样，ByteBuffer 中的数据确实以 NALU 标头开头，然后是不同大小的数据。 MediaMuxer 在计算帧数时似乎“看到”了数据中的帧。 那么这里有什么问题呢？
最低 API 为 21，我已使用运行 Android 5 的三星 Galaxy S4 和几台运行 Lineageos Oreo 和 Nougat 的设备进行了测试。
这是设置 MediaMuxer 的代码：
void setupMuxer(File f) throws IOException {
if (DEBUG) Log.d(TAG, "Setup Muxer: " + f.getAbsolutePath() +" can write: " + f.canWrite());
MediaFormat format = MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_AVC, decoderWidth, decoderHeight);
format.setInteger(MediaFormat.KEY_BIT_RATE, 4000000);
format.setInteger(MediaFormat.KEY_FRAME_RATE, 29);
format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);
format.setByteBuffer("csd-0", ByteBuffer.wrap(sps)); // sps and pps have been retrieved from the stream's NAL 7/8
format.setByteBuffer("csd-1", ByteBuffer.wrap(pps));
format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 1920 * 1080);
muxer = new MediaMuxer(f.getPath(), MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
videoTrack = muxer.addTrack(format);
muxer.start();
}
为每个（完整的）NALU 1 和 NALU 5 调用此方法：
void muxFrame(ByteBuffer buf, int frame) {
MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
bufferInfo.offset = buf.arrayOffset();
bufferInfo.size = buf.position() - bufferInfo.offset;
bufferInfo.flags = (buf.get(4) & 0x1f) == 5 ? MediaCodec.BUFFER_FLAG_KEY_FRAME : 0;
bufferInfo.presentationTimeUs = computePresentationTime(frame);
if (DEBUG)
Log.d(TAG, "muxFrame frame: " + frame + " size: " + bufferInfo.size + " NAL: " + (buf.get(4) & 0x1f) + " Flags: " + bufferInfo.flags + " PTS: " + bufferInfo.presentationTimeUs + " content: " + BitByteUtil.toByteString(buf.array(), buf.arrayOffset(), 8));
try {
muxer.writeSampleData(videoTrack, buf, bufferInfo);
} catch (Exception e) {
Log.w(TAG, "muxer failed", e);
} finally {
}
}
private static long computePresentationTime(int frameIndex) {
return 42 + frameIndex * 1000000 / FRAME_RATE;
}
如果 MediaMuxer 在消耗 100 个 NALU 后停止，这是我的输出。
05.651 D/VideoDecoderView: Setup Muxer: /storage/emulated/0/Pictures/test.mpg can write: true
05.656 I/MPEG4Writer: limits: 4294967295/0 bytes/us, bit rate: -1 bps and the estimated moov size 3317 bytes
06.263 D/VideoDecoderView: muxFrame frame: 2 size: 7257 NAL: 5 Flags: 1 PTS: 66708 content: 0:000 1:000 2:000 3:001 4:101 5:184 6:000 7:015
06.264 I/MPEG4Writer: setStartTimestampUs: 66708
06.264 I/MPEG4Writer: Earliest track starting time: 66708
06.308 D/VideoDecoderView: muxFrame frame: 3 size: 8998 NAL: 1 Flags: 0 PTS: 100042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:034 7:020
06.342 D/VideoDecoderView: muxFrame frame: 4 size: 13664 NAL: 1 Flags: 0 PTS: 133375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:066 7:020
06.375 D/VideoDecoderView: muxFrame frame: 5 size: 13674 NAL: 1 Flags: 0 PTS: 166708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:098 7:020
06.409 D/VideoDecoderView: muxFrame frame: 6 size: 13772 NAL: 1 Flags: 0 PTS: 200042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:130 7:020
06.483 D/VideoDecoderView: muxFrame frame: 7 size: 13707 NAL: 1 Flags: 0 PTS: 233375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:162 7:020
06.520 D/VideoDecoderView: muxFrame frame: 8 size: 13778 NAL: 1 Flags: 0 PTS: 266708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:194 7:020
06.555 D/VideoDecoderView: muxFrame frame: 9 size: 13743 NAL: 1 Flags: 0 PTS: 300042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:226 7:020
06.575 D/VideoDecoderView: muxFrame frame: 10 size: 7338 NAL: 5 Flags: 1 PTS: 333375 content: 0:000 1:000 2:000 3:001 4:101 5:184 6:000 7:015
06.593 D/VideoDecoderView: muxFrame frame: 11 size: 9059 NAL: 1 Flags: 0 PTS: 366708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:034 7:020
06.618 D/VideoDecoderView: muxFrame frame: 12 size: 13587 NAL: 1 Flags: 0 PTS: 400042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:066 7:020
06.644 D/VideoDecoderView: muxFrame frame: 13 size: 13650 NAL: 1 Flags: 0 PTS: 433375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:098 7:020
06.671 D/VideoDecoderView: muxFrame frame: 14 size: 13797 NAL: 1 Flags: 0 PTS: 466708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:130 7:020
.... [snip]
09.620 D/VideoDecoderView: muxFrame frame: 97 size: 7212 NAL: 5 Flags: 1 PTS: 3233375 content: 0:000 1:000 2:000 3:001 4:101 5:184 6:000 7:015
09.661 D/VideoDecoderView: muxFrame frame: 98 size: 8814 NAL: 1 Flags: 0 PTS: 3266708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:034 7:020
09.692 D/VideoDecoderView: muxFrame frame: 99 size: 13566 NAL: 1 Flags: 0 PTS: 3300042 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:066 7:020
09.737 D/VideoDecoderView: muxFrame frame: 100 size: 13733 NAL: 1 Flags: 0 PTS: 3333375 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:098 7:020
09.771 D/VideoDecoderView: muxFrame frame: 101 size: 13771 NAL: 1 Flags: 0 PTS: 3366708 content: 0:000 1:000 2:000 3:001 4:065 5:224 6:130 7:020
09.775 D/MPEG4Writer: Video track stopping. Stop source
09.775 I/MPEG4Writer: Received total/0-length (100/1) buffers and encoded 100 frames. - Video
09.775 D/MPEG4Writer: Video track source stopping
09.775 D/MPEG4Writer: Video track source stopped
09.775 D/MPEG4Writer: Video track stopped. Stop source
09.775 D/MPEG4Writer: Stopping writer thread
09.776 D/MPEG4Writer: 0 chunks are written in the last batch
09.779 D/MPEG4Writer: Writer thread stopped
09.780 I/MPEG4Writer: Ajust the moov start time from 66708 us -> 66708 us
09.780 D/MPEG4Writer: Video track stopping. Stop source
【问题讨论】：
-
您的 MPEG4Writer 输出看起来与我从自己的代码中得到的非常相似，但显然您有 1 个 0 长度的缓冲区。换句话说，它似乎正在工作。你检查过
f.exists()返回假吗？只是 b/c 您将文件放在图片目录中并不意味着媒体扫描仪会找到它。 -
感谢一百万！有时“只见树木不见森林！”
标签： android h.264 android-mediacodec mediamuxer