以固定的步长将字节读入 Go 缓冲区

Question

我想一次读取和处理文件名给定的文件中的 1024 个字节。我不明白如何正确构造外循环，尤其是为了适应缓冲区将包含少于 1024 个字节的最终步幅

我尝试过的：

fs, _ := os.Open(filename)
defer fs.Close()
n := 1024 // 1kb
buff := make([]byte, n)
for {
    buff = make([]byte, n) // is this initialized correctly?
    n1, err := fs.Read(buff)
    if err != nil {
        if err == io.EOF {
            break
        }
        fmt.Println(err)
        break
    }
    fmt.Println("read n1 bytes...", n1)
    fmt.Println(buff)
}

我看过以下资源：

• Reading specific number of bytes from a buffered reader in golang
• Read arbitrary amount of bytes into buffer Golang
• How to read a binary file in Go
• Working with raw bytes from a network in go
• https://gobyexample.com/reading-files
• Writing into fixed size Buffers in Golang with offsets
• https://rwinslow.com/posts/use-flatbuffers-in-golang/
• reading file line by line in go
• How do I read in a large flat file in Golang
• https://golang.org/pkg/io/#ReadFull

Answer 1

以固定的步长将字节读入 Go 缓冲区

在文件名给定的文件中一次读取和处理 1024 个字节。

容纳缓冲区将包含更少的最终步幅 超过 1024 字节。

---

对于读取保证的字节数，请使用ioutil.ReadFull。对于高效的流读取，请使用bufio.Reader。为提高效率，只分配一次读取缓冲区，然后重复使用。

例如，

package main

import (
    "bufio"
    "fmt"
    "io"
    "io/ioutil"
    "os"
)

func main() {
    stride := 1024
    filename := testname(stride)

    f, err := os.Open(filename)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        return
    }
    defer f.Close()

    r := bufio.NewReader(f)
    buf := make([]byte, 0, stride)
    for {
        n, err := io.ReadFull(r, buf[:cap(buf)])
        buf = buf[:n]
        if err != nil {
            if err == io.EOF {
                break
            }
            if err != io.ErrUnexpectedEOF {
                fmt.Fprintln(os.Stderr, err)
                break
            }
        }

        fmt.Println("read n bytes...", n)
        // process buf
    }
}

func testname(stride int) string {
    f, err := ioutil.TempFile("", "test.stride.")
    if err != nil {
        panic(err)
    }
    defer f.Close()
    _, err = f.Write(make([]byte, 2*stride+stride/2))
    if err != nil {
        panic(err)
    }
    return f.Name()
}

游乐场：https://play.golang.org/p/iYOY-z7hkoz

输出：

read n bytes... 1024
read n bytes... 1024
read n bytes... 512