用于下载大文件的 Spring Boot Java 堆空间

Question

我正在尝试编写一个 REST api 以允许用户在 Spring 启动时下载大文件（即 > 2GB）。我遇到了“Java Heap outOfMemoryException”。我试图对问题进行分类，我看到 HttpServetResponse 对象的类型是：ContentCachingResponseWrapper。 此类缓存写入输出流的所有内容，当缓存的数据大小变为 258MB 左右时，我得到 OutOfMemoryException。为什么是 248 MB，因为 JVM 有 256 MB 的堆内存。

ContentCachingResponseWrapper 中的默认 flushBuffer() 方法为空。如果我尝试调用用于将数据从缓存复制到流的 copyBodyToResponse()，它可以正常工作，但它也会关闭流。这导致只向客户端发送第一块数据。

有什么建议吗？

public void myDownloader(HttpServletRequest request, HttpServletResponse response) {
             //response.getClass() is: ContentCachingResponseWrapper  
             byte[] buffer = new byte[1048576];  // 1 MB Chunks
             FileInputStream inputStream = new FileInputStream(PATH_TO_SOME_VALID_FILE);
             int bytesRead= 0;
             ServletOutputStream outputStream = response.getOutputStream();

             while ((bytesRead = inputStream.read(buffer)) != -1) {              
                 outputStream.write(buffer, 0, bytesRead);
                 outputStream.flush();
                 response.flushBuffer();
               } 
}

我收到以下错误：

Caused by: java.lang.OutOfMemoryError: Java heap space
    at org.springframework.util.FastByteArrayOutputStream.addBuffer(FastByteArrayOutputStream.java:303) ~[spring-core-5.2.8.RELEASE.jar!/:5.2.8.RELEASE]
    at org.springframework.util.FastByteArrayOutputStream.write(FastByteArrayOutputStream.java:118) ~[spring-core-5.2.8.RELEASE.jar!/:5.2.8.RELEASE]
    at org.springframework.web.util.ContentCachingResponseWrapper$ResponseServletOutputStream.write(ContentCachingResponseWrapper.java:239) ~[spring-web-5.2.8.RELEASE.jar!/:5.2.8.RELEASE]

Answer 1

这是下载文件的一种非常基本的方法，但是如果您从浏览器调用它，浏览器会将其显示在屏幕上，并且可能会一直旋转（如果您问我是浏览器问题）：

@RequestMapping(path = "/downloadLargeFile", method = RequestMethod.GET)
public ResponseEntity<Resource> downloadLargeFile() {
    final File file = new File("c:/large.bin");
    final FileSystemResource resource = new FileSystemResource(file);
    return ResponseEntity.ok().body(resource);
}

因此您可以包含一些带有文件信息的标题，浏览器会将其下载到您的下载目录中的文件中，并且不会旋转：

@RequestMapping(path = "/downloadLargeFile2", method = RequestMethod.GET)
public ResponseEntity<Resource> downloadLargeFile2() {
    final HttpHeaders httpHeaders = new HttpHeaders();
    final File file = new File("c:/large.bin");
    final FileSystemResource resource = new FileSystemResource(file);
    httpHeaders.set(HttpHeaders.LAST_MODIFIED, String.valueOf(file.lastModified()));
    httpHeaders.set(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + file.getName() + "\"");
    httpHeaders.set(HttpHeaders.CONTENT_LENGTH, String.valueOf(file.length()));
    return ResponseEntity.ok()
        .headers(httpHeaders)
        .contentLength(file.length())
        .contentType(MediaType.APPLICATION_OCTET_STREAM)
        .body(resource);
}

要分块响应，请使用 InputStreamResource，显然 Spring 会关闭 InputStream：

@RequestMapping(path = "/pub/storage/downloadLargeFile4", method = RequestMethod.GET)
public ResponseEntity<InputStreamResource> downloadLargeFile4()
    throws Exception {
    final HttpHeaders httpHeaders = new HttpHeaders();
    final File file = new File("c:/large.bin");
    final InputStream inputStream = new FileInputStream(file);
    final InputStreamResource resource = new InputStreamResource(inputStream);
    httpHeaders.set(HttpHeaders.LAST_MODIFIED, String.valueOf(file.lastModified()));
    httpHeaders.set(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + file.getName() + "\"");
    httpHeaders.set(HttpHeaders.CONTENT_LENGTH, String.valueOf(file.length()));
    return ResponseEntity.ok()
        .headers(httpHeaders)
        .contentLength(file.length())
        .contentType(MediaType.APPLICATION_OCTET_STREAM)
        .body(resource);
}

进口：

import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

我也发现这个帖子很有用： download a file from Spring boot rest service