将 h264 RTSP 流读入 python 和 opencv答案

【问题标题】：reading a h264 RTSP stream into python and opencv将 h264 RTSP 流读入 python 和 opencv
【发布时间】：2014-03-22 18:54:02
【问题描述】：

问题来了

我有一个使用 RTSP 协议流式传输 h264 视频的 IP 摄像机，我要做的就是读取此流并将其传递给 Open CV 进行解码，使用这个功能

cv2.imdecode()

怎么做？

更新

我解决了这个问题

这里是解决方案：Convert YUVj420p pixel format to RGB888 using gstreamer

【问题讨论】：

imdecode 用于图像。你需要使用cv2.VideoCapture
我会逐帧抓取并解码
VideoCapture 会为您做到这一点。您不能使用 imdecode 来做到这一点，这仅适用于图像（jpeg、png、tiff 等）。以我个人的经验，videocapture 在某些 rtsp 流中也不起作用，在这种情况下，您需要使用额外的层来解码帧并将其传递给 opencv。比如用vlc做视频解码，比如vlc.exe -I dummy --dummy-quiet rtsp://10.10.3.133/h264 :sout=#transcode{vcodec=MJPG,vb=5000,scale=1,acodec=none}:standard{access=http,mux=raw,dst=127.0.0.1:9080/frame.mjpg} vlc://quit
我现在正在使用 gstreamer，我已经能够读取和解码 ip camera 流你知道如何将流传递给 opencv
@ZawLin 感谢您的命令，虽然我必须将 m‌ux=raw 更改为 mux=ts 才能使其正常工作，但 raw 似乎不是一个有效的选项。

标签： python opencv gstreamer rtsp

【解决方案1】：

import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst
import numpy as np
import cv2

GObject.threads_init()
Gst.init(None)

def YUV_stream2RGB_frame(data):

    w=640
    h=368
    size=w*h

    stream=np.fromstring(data,np.uint8) #convert data form string to numpy array

    #Y bytes  will start form 0 and end in size-1 
    y=stream[0:size].reshape(h,w) # create the y channel same size as the image

    #U bytes will start from size and end at size+size/4 as its size = framesize/4 
    u=stream[size:(size+(size/4))].reshape((h/2),(w/2))# create the u channel  itssize=framesize/4

    #up-sample the u channel to be the same size as the y channel and frame using pyrUp func in opencv2
    u_upsize=cv2.pyrUp(u)

    #do the same for v channel 
    v=stream[(size+(size/4)):].reshape((h/2),(w/2))
    v_upsize=cv2.pyrUp(v)

    #create the 3-channel frame using cv2.merge func watch for the order
    yuv=cv2.merge((y,u_upsize,v_upsize))

    #Convert TO RGB format
    rgb=cv2.cvtColor(yuv,cv2.cv.CV_YCrCb2RGB)

    #show frame
    cv2.imshow("show",rgb)
    cv2.waitKey(5)

def on_new_buffer(appsink):

   sample = appsink.emit('pull-sample')
   #get the buffer
   buf=sample.get_buffer()
   #extract data stream as string
   data=buf.extract_dup(0,buf.get_size())
   YUV_stream2RGB_frame(data)
   return False

def Init():

   CLI="rtspsrc name=src location=rtsp://192.168.1.20:554/live/ch01_0 latency=10   !decodebin ! appsink name=sink"

   #simplest way to create a pipline
   pipline=Gst.parse_launch(CLI)

   #getting the sink by its name set in CLI
   appsink=pipline.get_by_name("sink")

   #setting some important properties of appsnik
   appsink.set_property("max-buffers",20) # prevent the app to consume huge part of memory
   appsink.set_property('emit-signals',True) #tell sink to emit signals
   appsink.set_property('sync',False) #no sync to make decoding as fast as possible

   appsink.connect('new-sample', on_new_buffer) #connect signal to callable func

def run():
    pipline.set_state(Gst.State.PLAYING)
    GObject.MainLoop.run()


Init()
run()

【讨论】：