如何通过 Java 中的 PrintWriter 将数据分块写入 csv

Question

我正在处理一些数据，然后将其保存在 stringbuilder 中以最终写入 csv。

目前，我正在使用作为处理一些 idData 数组的结果获得的 println 一次编写整个 stringBuilder。但是我想分批写，所以也许假设ids的数组数据长度= 200，我想一次写50个id的数据。

我该怎么做？

Existing code of writing the entire string builder result

class A {
 private StringBuilder sb;

 public void appendBody(String[] idData) {
   for(String id: idData) {
     Data data = getData(id);
     processData(data);
     
   }
 writeToCsv();
 } 

 public void processData() {

  .. processing that involves populating private property string builder
  // adding some sample data here
  this.sb.append("Name,Phone,Email\n");
  this.sb.append("Glen,1234,glen@abcd.com\n");
  this.sb.append("John,5678,john@efgh.com");

  // end populating string builder
 }

 public String writeToCsv() throws DataNotFoundException {

   String filename = "outputFileNamePath";  // sample filename, modified to be pasted here
   PrintWriter writer;
     try {
         writer = new PrintWriter(new File(filename));
     } catch (FileNotFoundException e) {
         writeLog("ERROR", e.getMessage);
         throw new Exception("Error while creating output file");
     }
     writer.println(this.sb.toString());
     writer.close();
     return filename;
  }
}

一些建议和提示会有所帮助。

P.S：请不要评论被重新抛出的日志消息或异常或变量的名称或任何与我的问题无关的细节，因为它们已被修改为将其添加到此处并请原谅我的任何错误提前！

Answer 1

这是您问题的解决方案。

基本思想是控制对 writeToCsv() 的调用（这将等于所需的块大小）并始终以附加模式打开文件

在这里，我使用 mod 操作来控制对 writeToCsv() 的调用。您可以添加任何符合您的卡盘尺寸要求的逻辑。文本将在 5 次迭代后推送到文件。

另外，要以附加模式打开文件，请使用

FileWriter fileWriter = new FileWriter(filename, true); // Set true for append mode
writer = new PrintWriter(fileWriter);

这是完整的程序。

public class WriteToFileInChunk {
    private static StringBuilder sb = new StringBuilder();

    public static void main(String[] args) throws IOException {
        WriteToFileInChunk writeToFileInChunk = new WriteToFileInChunk();
        writeToFileInChunk.writeInchuncks();
    }

    private void writeInchuncks() throws IOException {
        String[] idData = { "1,", "2,", "3,", "4,", "5,", "6,", "7,", "8,", "9,", "10,", "11,", "12,", "13,", "14,",
                "15,", "16,", "17,", "18,", "19,", "20" };

        for (int i = 0; i < idData.length; i++) {
            StringBuilder data = new StringBuilder(idData[i]);
            sb.append(processData(data));

            if (i % 5 == 0) {
                writeToCsv(sb);
                sb = new StringBuilder();
            }
        }
    }

    private StringBuilder processData(StringBuilder stringBuilder) {
        return stringBuilder.append(System.currentTimeMillis() + ", Name, Phone,Email\n");
    }

    private String writeToCsv(StringBuilder sb) throws IOException {

        String filename = "outputFileNamePath"; // sample filename, modified to be pasted here
        PrintWriter writer;
        try {

            FileWriter fileWriter = new FileWriter(filename, true); // Set true for append mode
            writer = new PrintWriter(fileWriter);
            writer.println(sb.toString());
            writer.close();

        } catch (FileNotFoundException e) {
            System.out.printf("ERROR : %s", e.getMessage());
            new Exception("Error while creating output file", e);
        }
        return filename;
    }
}

Answer 2

您需要将idData 拆分为多批数组。然后writeToCsv处理完每批。 （注意你必须清除sb）

public void appendBody(String[] idData) {
    for (String[] batch : split(idData, BATCH_SIZE)) {
        sb = new StringBuilder();
        for (String id : batch) {
            Data data = getData(id);
            processData(data);
        }
        writeToCsv();
    }
}

您可以使用以下代码来拆分数组：

static List<String[]> split(String[] idData, int batchSize) {
    List<String[]> list = new ArrayList<>();
    for (int i = 0; i < idData.length; i += batchSize) {
        int realBatchSize = Math.min(batchSize, idData.length - i);
        String[] batch = new String[realBatchSize];
        for (int j = 0; j < realBatchSize; j++)
            batch[j] = idData[j + i];
        list.add(batch);
    }
    return list;
}

记得在打开文件时使用APPEND 模式。