使用 gstreamer / gstreamer-java 在 mjpeg 流的两个 jpeg 帧之间解析文本数据

Question

我创建了一个 mjpeg 流。我在两个 jpeg 帧之间[在第一帧的页脚和下一帧的页眉之间]嵌入了内容长度。

[MJPEG STREAM]
      |
      V
...
----------
JPEG FRAME
(Image)
----------
Content-length
(Text)
----------
JPEG FRAME
(Image)
----------
Content-length
(Text)
----------
JPEG FRAME
(Image)
----------
...
      |
      V

我可以使用 gstreamer-java、gstreamer C API 和 gst-launch 播放视频。但是我正在尝试解析 gstreamer-java 中的内容长度文本。我尝试了 "meta tags" 但这并没有产生我需要的任何结果

我按如下方式播放我创建的 mjpeg 流

gst-launch -v souphttpsrc location="<ip>:<port>/<cgi_bin folder>/<name>.cgi" do-timestamp=true is_live=true ! multipartdemux ! jpegdec ! ffmpegcolorspace ! autovideosink

我尝试使用元标记

gst-launch -v souphttpsrc location="<ip>:<port>/<cgi_bin folder>/<name>.cgi" do-timestamp=true is_live=true ! multipartdemux ! jpegdec ! fakesink -t

还有这个

gst-launch -v souphttpsrc location="<ip>:<port>/<cgi_bin folder>/<name>.cgi" do-timestamp=true is_live=true ! fakesink -t

然而，这个媒体管道的输出类似于以下内容

/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = "chain   ******* < (557568   
bytes, timestamp: 0:00:06.895505695, duration: none, offset: -1, offset_end: -1, flags: 
0) 0x9a82818"
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = "chain   ******* < (557568 
bytes, timestamp: 0:00:06.941328354, duration: none, offset: -1, offset_end: -1, flags: 
0) 0x9a6f6e0"

可以看出没有内容长度。 :( 。我使用 WireShark 并查看包含内容长度的数据包。随后我编写了一个单独的 Java 代码（没有 gstreamer 插件），我可以解析流以获取内容长度。但这不是我想要的理想方法与保持同步的确切帧相对应的内容长度。

我之前为 C API 代码写过 tees。我想知道 tee 是否适用于这种情况！

Answer 1

Appsink 派上了用场。管道看起来像

                         _ queue -> appsink -> parser 
                       /  
        video -> tee ->
                       \_ demux -> decoder -> ffmpegcolorspace -> autovideosink 

Java 代码 sn-p 的样子

首先定义appsink bin

final AppSink appsink = (AppSink) ElementFactory.make("appsink", "appsink");

那么它的属性...

appsink.set("emit-signals",true);
appsink.set("max-buffers",1);
//appsink.set("drop",true); //Make this non-blocking. 
                            //Meaning drop frames when queue is full

那么它的功能...

appsink.connect(new AppSink.NEW_BUFFER() {
@Override public void newBuffer(Element elem, Pointer userData) {
AppSink appsink = (AppSink) elem;
Buffer buffer = appsink.pullBuffer();  //get the stream data as an input to external 
                                       //applications

/*
// Create a character ByteBuffer
CharBuffer cbuf = byteBuffer.asCharBuffer();
*/

ByteBuffer byteBuffer = buffer.getByteBuffer();
byte[] bytearray = new byte[byteBuffer.remaining()];
byteBuffer.get(bytearray);
String s = new String(bytearray);
//InputStream is = new ByteArrayInputStream(bytearray);
// System.out.print(buffer.getSize()+"\n");
//  System.out.print(s+"\n");

//Do whatever with this buffer inside string here e.g. parsing text content

...

byteBuffer.clear();
buffer.dispose();
buffer = null;

}
});           

//=============================================== ===============================

C 代码 sn-p 的样子

...
sink = gst_element_factory_make ("appsink", "Output");
g_assert (sink);

那么……

...
gst_app_sink_set_max_buffers((GstAppSink*)sink, 100);
gst_app_sink_set_emit_signals ((GstAppSink*)sink, TRUE);
g_signal_connect (sink, "new-buffer",  G_CALLBACK(new_buffer), NULL);

那么……

static GstFlowReturn new_buffer (GstAppSink *app_sink, gpointer user_data)
{
  char* pipe_name = (char*) user_data;
  GstBuffer* buffer = gst_app_sink_pull_buffer(app_sink);

//  g_debug("appsink buffer timestamp(%lli) size(%d)",
//          GST_BUFFER_TIMESTAMP(buffer),
//          GST_BUFFER_SIZE(buffer));

  g_print((char*)buffer);

  gst_buffer_unref(buffer);
  return GST_FLOW_OK;
}