如何正确处理 tcp 流并在 python 中按行拆分？答案

【问题标题】：How to correctly handle tcp stream and split by lines in python?如何正确处理 tcp 流并在 python 中按行拆分？
【发布时间】：2022-07-25 02:59:53
【问题描述】：

我正在使用下面的代码来处理来自 tcp 连接的日期。但有时，在较慢的连接上，数据没有正确地分成几行，我用我想要处理的一半行调用了 handle_line()。该行的第二部分被称为下一个。任何想法如何解决这个问题？这用于 HTTP，因此每一行都以 \r\n 结尾。

from io import BytesIO
import time
import logging
import errno
import threading
import socket
import os

def listener(client, address):
   try:
      with BytesIO() as buffer:
         while True:
            time.sleep(0.5) #slow down for low speed of data transmission to get fill lines, still does not solve
            resp = client.recv(1024)
            if not resp:
               logging.info('Connection closed by ' + str(address))
               break
            else:
               buffer.write(resp)          # Write to the BytesIO object
               buffer.seek(0)              # Set the file pointer to the SoF
               start_index = 0             # Count the number of characters processed
               for line in buffer:
                   start_index += len(line)
                   handle_line(client,line)       # Do something with your line
               if start_index:
                   buffer.seek(start_index)
                   remaining = buffer.read()
                   buffer.truncate(0)
                   buffer.seek(0)
                   buffer.write(remaining)
               else:
                   buffer.seek(0, 2)
   except IOError as e:
      if e.errno == errno.EPIPE:
         logging.warning(e)
      else:
         logging.warning(e)
   finally:
      client.close()


class nc(threading.Thread):
   def run(self):
      s = socket.socket()
      s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
      try:
         s.bind((tcp_host,tcp_port))
      except:
         logging.error('Can not bind address')
         os._exit(1)
      s.listen(1)
      th = []
      while True:
         client, address = s.accept()
         logging.info('Incoming connection from ' + str(address))
         th.append(threading.Thread(target=listener, args = (client,address)).start())
      s.close()


tcp_host = 'localhost'
tcp_port = 8080

server = nc()
server.daemon = True
server.start()

【问题讨论】：

标签： python tcp stream line handle

【解决方案1】：

这就是 TCP 的工作原理。如果你想要完整的消息，缓冲数据流直到你有一个完整的消息，然后从缓冲区中提取它。套接字有一个内置的“类文件”包装器，它具有您可以使用的.readline() 方法。还可以使用with 自动关闭阅读器和客户端对象。

未经测试：

def listener(client, address):
    with client, client.makefile('rb') as reader:
        try:
            while True:
                resp = reader.readline()
                if not resp:
                   logging.info('Connection closed by ' + str(address))
                   break
                handle_line(client,line)       # Do something with your line
        except IOError as e:
            if e.errno == errno.EPIPE:
                logging.warning(e)
            else:
                logging.warning(e)

【讨论】：