【问题标题】:How do I concatenate two System.IO.Stream instances into one?如何将两个 System.Io.Stream 实例连接成一个?
【发布时间】:2011-04-22 04:54:43
【问题描述】:

假设我想将三个文件连续流式传输给一个用户,但不是他递给我一个 Stream 对象来推送字节,我必须递给他一个 Stream 对象他会拉字节从。我想使用我的三个FileStream 对象(或者更聪明的IEnumerable<Stream>)并返回一个新的ConcatenatedStream 对象,该对象可以按需从源流中提取。

【问题讨论】:

    标签: c# .net stream


    【解决方案1】:
    class ConcatenatedStream : Stream
    {
        Queue<Stream> streams;
    
        public ConcatenatedStream(IEnumerable<Stream> streams)
        {
            this.streams = new Queue<Stream>(streams);
        }
    
        public override bool CanRead
        {
            get { return true; }
        }
    
        public override int Read(byte[] buffer, int offset, int count)
        {
            int totalBytesRead = 0;
    
            while (count > 0 && streams.Count > 0)
            {
                int bytesRead = streams.Peek().Read(buffer, offset, count);
                if (bytesRead == 0)
                {
                    streams.Dequeue().Dispose();
                    continue;
                }
    
                totalBytesRead += bytesRead;
                offset += bytesRead;
                count -= bytesRead;
            }
    
            return totalBytesRead;
        }
    
        public override bool CanSeek
        {
            get { return false; }
        }
    
        public override bool CanWrite
        {
            get { return false; }
        }
    
        public override void Flush()
        {
            throw new NotImplementedException();
        }
    
        public override long Length
        {
            get { throw new NotImplementedException(); }
        }
    
        public override long Position
        {
            get
            {
                throw new NotImplementedException();
            }
            set
            {
                throw new NotImplementedException();
            }
        }
    
        public override long Seek(long offset, SeekOrigin origin)
        {
            throw new NotImplementedException();
        }
    
        public override void SetLength(long value)
        {
            throw new NotImplementedException();
        }
    
        public override void Write(byte[] buffer, int offset, int count)
        {
            throw new NotImplementedException();
        }
    }
    

    【讨论】:

    • 我想知道。也许解决方案是如此优雅以至于它是非法的。
    • 我想知道您为什么使用堆栈而不是队列?或者,可以通过存储 Enumerator 而不是在一开始就枚举 IEnumerable 来使其更加懒惰 - 这也具有恒定空间的优点。
    • (comment-by-proxy, ie "not me") "这个解决方案基本上不起作用,并且包含一个严重的错误。如果我尝试一次读取所有数据,我将只获得第一个流数据。”
    • 错误是bytesRead &lt; count 不是关闭队列头部的流的充分理由。它必须是。因此,如果您要连接在固定大小块中运行的流(例如某些情况下的 CryptoStream),则流将在完全读取之前关闭,除非count 与块大小完全匹配。
    • 这个解决方案的思路很好,但是这个实现真的很破。鉴于其他答案,我也不明白为什么会接受这一点。任何人都不应使用此代码。
    【解决方案2】:

    未经测试,但类似:

    class StreamEnumerator : Stream
    {
        private long position;
        bool closeStreams;
        IEnumerator<Stream> iterator;
        Stream current;
        private void EndOfStream() {
            if (closeStreams && current != null)
            {
                current.Close();
                current.Dispose();
            }
            current = null;
        }
        private Stream Current
        {
            get {
                if(current != null) return current;
                if (iterator == null) throw new ObjectDisposedException(GetType().Name);
                if (iterator.MoveNext()) {
                    current = iterator.Current;
                }
                return current;
            }
        }
        protected override void Dispose(bool disposing)
        {
            if (disposing)
            {
                EndOfStream();
                iterator.Dispose();
                iterator = null;
                current = null;
            }
            base.Dispose(disposing);
        }
        public StreamEnumerator(IEnumerable<Stream> source, bool closeStreams)
        {
            if (source == null) throw new ArgumentNullException("source");
            iterator = source.GetEnumerator();
            this.closeStreams = closeStreams;
        }
        public override bool CanRead { get { return true; } }
        public override bool CanWrite { get { return false; } }
        public override void Write(byte[] buffer, int offset, int count)
        {
            throw new NotSupportedException();
        }
        public override void WriteByte(byte value)
        {
            throw new NotSupportedException();
        }
        public override bool CanSeek { get { return false; } }
        public override bool CanTimeout { get { return false; } }
        public override void SetLength(long value)
        {
            throw new NotSupportedException();
        }
        public override long Seek(long offset, SeekOrigin origin)
        {
            throw new NotSupportedException();
        }
        public override void Flush()
        { /* nothing to do */ }
        public override long Length
        {
            get { throw new NotSupportedException(); }
        }
        public override long Position
        {
            get { return position; }
            set { if (value != this.position) throw new NotSupportedException(); }
        }
        public override int Read(byte[] buffer, int offset, int count)
        {
            int result = 0;
            while (count > 0)
            {
                Stream stream = Current;
                if (stream == null) break;
                int thisCount = stream.Read(buffer, offset, count);
                result += thisCount;
                count -= thisCount;
                offset += thisCount;
                if (thisCount == 0) EndOfStream();
            }
            position += result;
            return result;
        }
    }
    

    【讨论】:

      【解决方案3】:

      只要你只需要阅读,这是我对这样一个流的实现:

      注意! Position and Seek 坏了,需要修复

      using System;
      using System.Collections.Generic;
      using System.Diagnostics;
      using System.IO;
      
      namespace LVK.IO
      {
          /// <summary>
          /// This class is a <see cref="Stream"/> descendant that manages multiple underlying
          /// streams which are considered to be chained together to one large stream. Only reading
          /// and seeking is allowed, writing will throw exceptions.
          /// </summary>
          public class CombinedStream : Stream
          {
              private readonly Stream[] _UnderlyingStreams;
              private readonly Int64[] _UnderlyingStartingPositions;
              private Int64 _Position;
              private readonly Int64 _TotalLength;
              private int _Index;
      
              /// <summary>
              /// Constructs a new <see cref="CombinedStream"/> on top of the specified array
              /// of streams.
              /// </summary>
              /// <param name="underlyingStreams">
              /// An array of <see cref="Stream"/> objects that will be chained together and
              /// considered to be one big stream.
              /// </param>
              public CombinedStream(params Stream[] underlyingStreams)
              {
                  if (underlyingStreams == null)
                      throw new ArgumentNullException("underlyingStreams");
                  foreach (Stream stream in underlyingStreams)
                  {
                      if (stream == null)
                          throw new ArgumentNullException("underlyingStreams[]");
                      if (!stream.CanRead)
                          throw new InvalidOperationException("CanRead not true for all streams");
                      if (!stream.CanSeek)
                          throw new InvalidOperationException("CanSeek not true for all streams");
                  }
      
                  _UnderlyingStreams = new Stream[underlyingStreams.Length];
                  _UnderlyingStartingPositions = new Int64[underlyingStreams.Length];
                  Array.Copy(underlyingStreams, _UnderlyingStreams, underlyingStreams.Length);
      
                  _Position = 0;
                  _Index = 0;
      
                  _UnderlyingStartingPositions[0] = 0;
                  for (int index = 1; index < _UnderlyingStartingPositions.Length; index++)
                  {
                      _UnderlyingStartingPositions[index] =
                          _UnderlyingStartingPositions[index - 1] +
                          _UnderlyingStreams[index - 1].Length;
                  }
      
                  _TotalLength =
                      _UnderlyingStartingPositions[_UnderlyingStartingPositions.Length - 1] +
                      _UnderlyingStreams[_UnderlyingStreams.Length - 1].Length;
              }
      
              /// <summary>
              /// Gets a value indicating whether the current stream supports reading.
              /// </summary>
              /// <value>
              /// <c>true</c>.
              /// </value>
              /// <returns>
              /// Always <c>true</c> for <see cref="CombinedStream"/>.
              /// </returns>
              public override Boolean CanRead
              {
                  get
                  {
                      return true;
                  }
              }
      
              /// <summary>
              /// Gets a value indicating whether the current stream supports seeking.
              /// </summary>
              /// <value>
              /// <c>true</c>.
              /// </value>
              /// <returns>
              /// Always <c>true</c> for <see cref="CombinedStream"/>.
              /// </returns>
              public override Boolean CanSeek
              {
                  get
                  {
                      return true;
                  }
              }
      
              /// <summary>
              /// Gets a value indicating whether the current stream supports writing.
              /// </summary>
              /// <value>
              /// <c>false</c>.
              /// </value>
              /// <returns>
              /// Always <c>false</c> for <see cref="CombinedStream"/>.
              /// </returns>
              public override Boolean CanWrite
              {
                  get
                  {
                      return false;
                  }
              }
      
              /// <summary>
              /// When overridden in a derived class, clears all buffers for this stream and causes any buffered data to be written to the underlying device.
              /// </summary>
              /// <exception cref="T:System.IO.IOException">An I/O error occurs. </exception>
              public override void Flush()
              {
                  foreach (Stream stream in _UnderlyingStreams)
                  {
                      stream.Flush();
                  }
              }
      
              /// <summary>
              /// Gets the total length in bytes of the underlying streams.
              /// </summary>
              /// <value>
              /// The total length of the underlying streams.
              /// </value>
              /// <returns>
              /// A long value representing the total length of the underlying streams in bytes.
              /// </returns>
              /// <exception cref="T:System.NotSupportedException">A class derived from Stream does not support seeking. </exception>
              /// <exception cref="T:System.ObjectDisposedException">Methods were called after the stream was closed. </exception>
              public override Int64 Length
              {
                  get
                  {
                      return _TotalLength;
                  }
              }
      
              /// <summary>
              /// Gets or sets the position within the current stream.
              /// </summary>
              /// <value></value>
              /// <returns>The current position within the stream.</returns>
              /// <exception cref="T:System.IO.IOException">An I/O error occurs. </exception>
              /// <exception cref="T:System.NotSupportedException">The stream does not support seeking. </exception>
              /// <exception cref="T:System.ObjectDisposedException">Methods were called after the stream was closed. </exception>
              public override Int64 Position
              {
                  get
                  {
                      return _Position;
                  }
      
                  set
                  {
                      if (value < 0 || value > _TotalLength)
                          throw new ArgumentOutOfRangeException("Position");
      
                      _Position = value;
                      if (value == _TotalLength)
                      {
                          _Index = _UnderlyingStreams.Length - 1;
                          _Position = _UnderlyingStreams[_Index].Length;
                      }
      
                      else
                      {
                          while (_Index > 0 && _Position < _UnderlyingStartingPositions[_Index])
                          {
                              _Index--;
                          }
      
                          while (_Index < _UnderlyingStreams.Length - 1 &&
                                 _Position >= _UnderlyingStartingPositions[_Index] + _UnderlyingStreams[_Index].Length)
                          {
                              _Index++;
                          }
                      }
                  }
              }
      
              /// <summary>
              /// Reads a sequence of bytes from the current stream and advances the position within the stream by the number of bytes read.
              /// </summary>
              /// <param name="buffer">An array of bytes. When this method returns, the buffer contains the specified byte array with the values between offset and (offset + count - 1) replaced by the bytes read from the current source.</param>
              /// <param name="offset">The zero-based byte offset in buffer at which to begin storing the data read from the current stream.</param>
              /// <param name="count">The maximum number of bytes to be read from the current stream.</param>
              /// <returns>
              /// The total number of bytes read into the buffer. This can be less than the number of bytes requested if that many bytes are not currently available, or zero (0) if the end of the stream has been reached.
              /// </returns>
              /// <exception cref="T:System.ArgumentException">The sum of offset and count is larger than the buffer length. </exception>
              /// <exception cref="T:System.ObjectDisposedException">Methods were called after the stream was closed. </exception>
              /// <exception cref="T:System.NotSupportedException">The stream does not support reading. </exception>
              /// <exception cref="T:System.ArgumentNullException">buffer is null. </exception>
              /// <exception cref="T:System.IO.IOException">An I/O error occurs. </exception>
              /// <exception cref="T:System.ArgumentOutOfRangeException">offset or count is negative. </exception>
              public override int Read(Byte[] buffer, int offset, int count)
              {
                  int result = 0;
                  while (count > 0)
                  {
                      _UnderlyingStreams[_Index].Position = _Position - _UnderlyingStartingPositions[_Index];
                      int bytesRead = _UnderlyingStreams[_Index].Read(buffer, offset, count);
                      result += bytesRead;
                      offset += bytesRead;
                      count -= bytesRead;
                      _Position += bytesRead;
      
                      if (count > 0)
                      {
                          if (_Index < _UnderlyingStreams.Length - 1)
                              _Index++;
                          else
                              break;
                      }
                  }
      
                  return result;
              }
      
              /// <summary>
              /// Sets the position within the current stream.
              /// </summary>
              /// <param name="offset">A byte offset relative to the origin parameter.</param>
              /// <param name="origin">A value of type <see cref="T:System.IO.SeekOrigin"></see> indicating the reference point used to obtain the new position.</param>
              /// <returns>
              /// The new position within the current stream.
              /// </returns>
              /// <exception cref="T:System.IO.IOException">An I/O error occurs. </exception>
              /// <exception cref="T:System.NotSupportedException">The stream does not support seeking, such as if the stream is constructed from a pipe or console output. </exception>
              /// <exception cref="T:System.ObjectDisposedException">Methods were called after the stream was closed. </exception>
              public override long Seek(long offset, SeekOrigin origin)
              {
                  switch (origin)
                  {
                      case SeekOrigin.Begin:
                          Position = offset;
                          break;
      
                      case SeekOrigin.Current:
                          Position += offset;
                          break;
      
                      case SeekOrigin.End:
                          Position = Length + offset;
                          break;
                  }
      
                  return Position;
              }
      
              /// <summary>
              /// Throws <see cref="NotSupportedException"/> since the <see cref="CombinedStream"/>
              /// class does not supports changing the length.
              /// </summary>
              /// <param name="value">The desired length of the current stream in bytes.</param>
              /// <exception cref="T:System.NotSupportedException">
              /// <see cref="CombinedStream"/> does not support this operation.
              /// </exception>
              public override void SetLength(long value)
              {
                  throw new NotSupportedException("The method or operation is not supported by CombinedStream.");
              }
      
              /// <summary>
              /// Throws <see cref="NotSupportedException"/> since the <see cref="CombinedStream"/>
              /// class does not supports writing to the underlying streams.
              /// </summary>
              /// <param name="buffer">An array of bytes.  This method copies count bytes from buffer to the current stream.</param>
              /// <param name="offset">The zero-based byte offset in buffer at which to begin copying bytes to the current stream.</param>
              /// <param name="count">The number of bytes to be written to the current stream.</param>
              /// <exception cref="T:System.NotSupportedException">
              /// <see cref="CombinedStream"/> does not support this operation.
              /// </exception>
              public override void Write(byte[] buffer, int offset, int count)
              {
                  throw new NotSupportedException("The method or operation is not supported by CombinedStream.");
              }
          }
      }
      

      【讨论】:

      • 很好,来自现有的图书馆。我认为这是对我提出的问题的一个很好的一般答案,尽管我更喜欢不急切地评估流的东西,因为我更喜欢按需构建它们。 LVK 正在抓取所有这些以支持有效的窥视,这是有道理的,尽管就我而言,我不必支持该操作。
      • 在这种情况下你应该使用@Hasan Khanhere提供的那个。
      • 应添加处理 UnderlyingStreams 的 Dispose 方法的实现
      • 链接失效了。这个答案不是问题,因为完整的代码在这里,但我注意到你没有嵌入代码的其他答案(例如this one)。
      • 我已经为它创建了一个 GitHub 项目 - github.com/lassevk/Streams - 很快就会提交一些测试和修复代码(不过可能不会在周末之前)
      【解决方案4】:

      编辑:明确表示这是一个可搜索的选项。

      这是一个支持搜索的选项,这在许多情况下都是必要的。它缺少一些用于处理底层流的功能,如果您愿意假设底层流不会改变长度,它可能会更有效率。然后可以计算一次总长度和流偏移量。

      public sealed class SeekableConcatenatedStream : Stream
      {
          List<Stream> streams;
          private long _Position { get; set; }
      
          public SeekableConcatenatedStream(List<Stream> streams)
          {
              foreach (var s in streams)
              {
                if (!s.CanSeek)
                    throw new ArgumentException($"All provided streams must be be seekable to create a {nameof(SeekableConcatenatedStream)}");
              }
      
              this.streams = streams;
              Seek(0, SeekOrigin.Begin);
          }
      
          public override bool CanRead
          {
              get { return true; }
          }
      
          public override int Read(byte[] buffer, int offset, int count)
          {
              if (streams.Count == 0)
                  return 0;
      
              var startStream = 0;
              var cumulativeCapacity = 0L;
              for (var i = 0; i < streams.Count; i++)
              {
                  cumulativeCapacity += streams[i].Length;
                  if (_Position < cumulativeCapacity)
                  {
                      startStream = i;
                      break;
                  }
              }
      
              var bytesRead = 0;
              var curStream = startStream;
      
              while (_Position < Length && bytesRead < count && curStream < streams.Count)
              {
                  var r = streams[curStream].Read(buffer, offset + bytesRead, count - bytesRead);
                  bytesRead += r;
                  Seek(_Position + r, SeekOrigin.Begin);
                  curStream++;
              }
      
              return bytesRead;
          }
      
          public override bool CanSeek
          {
              get { return true; }
          }
      
          public override bool CanWrite
          {
              get { return false; }
          }
      
          public override void Flush()
          {
              throw new NotImplementedException();
          }
      
          public override long Length
          {
              get {
                  long length = 0;
                  for (var i = 0; i < streams.Count; i++)
                  {
                      length += streams[i].Length;
                  }
                  return length;
              }
          }
      
          public override long Position
          {
              get
              {
                  return _Position;
              }
              set
              {
                  Seek(value, SeekOrigin.Begin);
              }
          }
      
          public override long Seek(long offset, SeekOrigin origin)
          {
              if (origin == SeekOrigin.Begin)
              {
                  _Position = offset;
      
                  var prevLength = 0L;
                  var cumulativeLength = 0L;
                  for (var i = 0; i < streams.Count; i++)
                  {
                      cumulativeLength += streams[i].Length;
                      if (offset < cumulativeLength)
                      {
                          streams[i].Seek(offset - prevLength, SeekOrigin.Begin);
                          return _Position;
                      }
                      prevLength = cumulativeLength;
                  }
              }
      
              if (origin == SeekOrigin.Current)
              {
                  var newAbs = _Position + offset;
                  return Seek(newAbs, SeekOrigin.Begin);
              } 
              else if(origin == SeekOrigin.End)
              {
                  var newAbs = Length - offset;
                  return Seek(newAbs, SeekOrigin.Begin);
              }
      
              throw new NotImplementedException();
          }
      
          public override void SetLength(long value)
          {
              throw new NotImplementedException();
          }
      
          public override void Write(byte[] buffer, int offset, int count)
          {
              throw new NotImplementedException();
          }
      }
      

      【讨论】:

      • 这不是一个好的解决方案,因为并非所有Stream 实现都提供搜索功能(因此并非所有流都提供Length)。您的代码广泛依赖于可用的Length,因此它仅适用于支持搜索的流。
      • 我试图通过声明 Here is an option that supports seeking 将免责声明放在前面。可以完全合理地假设存在依赖可搜索流来解决的问题。如果你想在这样的上下文中提供一个连接流,它也需要是可搜索的。因此,说它“不是一个好的解决方案”是不公平的。您的观点提出了一个合理的改进,即如果任何子流不可搜索,则构建可搜索的级联流应该失败。如果我有时间,我会更新代码以反映它。
      • 当然可以连接不可搜索的流,正如接受的答案所证明的那样。在这种情况下,您添加CanSeek 至少会使您的答案在更好的时间点失败,因此我将删除我的反对票。
      【解决方案5】:

      为什么不使用已经封装了多个文件概念的容器,比如使用 ZipOutputStream from SharpZipLib

      【讨论】:

      • 很公平,虽然实际上我发送的数据不是文件——我正在动态构建它,所以我希望尽可能少地保存在内存中。
      • 是否发送文件无关紧要;您只需将您关心的流写入ZipOutputStream,并在每个流前面加上ZipEntry 标记
      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2016-03-27
      • 2014-05-21
      • 2020-02-27
      • 2012-12-19
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多