总数据未填写在nodejs中的exceljs excel表中答案

【问题标题】：Total data is not getting filled in exceljs excel sheet in nodejs总数据未填写在nodejs中的exceljs excel表中
【发布时间】：2022-02-22 12:09:50
【问题描述】：

我有一个报告任务，其中所有数据都应该填写在一个 excel 文件中，并且应该通过邮件从后端发送给客户。我已经使用 excel.js 编写了一个 excel 文件，并且使用较少的数据可以正常工作。如果数据更像 2000 或更多，那么所有数据都不会被填充到 excel 文件中。以下是我尝试过的示例。

下面是其中的API。

router.get('/:type/:fromDate/:toDate',userAuth,(req,res)=>{
    if(!req.query.ids) return res.send({'message':'Please send ID as query',statusCode:2});
    let ids = req.query.ids.split(',');
    var workbook = new Excel.Workbook();
    let type = req.params.type
    workbook.creator = ' 32';
    workbook.lastModifiedBy = '321';
    workbook.created = new Date();
    workbook.modified = new Date();
    workbook.views = [{
            x: 0, y: 0, width: 10000, height: 20000,
            firstSheet: 0, activeTab: 1, visibility: 'visible'
    }]
    var reportWorkSheet = workbook.addWorksheet( req.params.type +' Report', {
        pageSetup: { paperSize: 9, orientation: 'landscape' }
    });
    if(type === 'customers'){
        userCustomerReport(req,res ,ids , reportWorkSheet ,workbook );
    } else if(type === 'interactions'){
        userInteractionReport(req ,res, ids , reportWorkSheet ,workbook , req.params.fromDate , req.params.toDate);
    } else if(type === 'allocations'){
        userAllocationReport(req ,res,ids , reportWorkSheet ,workbook);
    } else return res.send({'message':'Please check the request type',statusCode:2});
})



commonColomns = () => ([
    { header: 'Customer Name', key: 'cName', width: 25, style: { font: { size: 12 } } },
    { header: 'Customer Phone', key: 'cPhone', width: 35, style: { font: { size: 12 } } },
    { header: 'Customer Email', key: 'cEmail', width: 35, style: { font: { size: 12 } } },
    { header: 'Customer Company Name', key: 'cCompName', width: 18, style: { font: { size: 12 } } },
    { header: 'Assigned to name', key: 'assignedTName', width: 18, style: { font: { size: 12 } } },
    { header: 'Assigned from name ', key: 'assignedFName', width: 20, style: { font: { size: 12 } } }
]);


// here i am generation all JSON data .

function  userInteractionReport(req ,res , ids ,reportWorkSheet , workbook , fromDate , toDate) {
    let idString = req.query.ids.split(',');
    let id =[];
    idString.forEach(element => {id.push(new ObjectID(element));});
    Interaction.aggregate([
        { $match:{$or: [{"assigned.toId":{$in:id}},{"assigned.fromId":{$in:id}}] ,createdTimeStamp : {$gte:Number(fromDate),$lt:Number(toDate)}} },
        { "$project": {
            "assigned": 1,
            "type": 1,
            "priority": 1,
            "customer": 1,
            "customFields": 1,
            "dateTime": 1,
            "notes":1,
            "length": { "$size": "$customFields" }
        }},
        { "$sort": { "length": -1 } },
    ])
    .then((interactions)=>{
        if(!interactions[0]){
            return res.send({'message':'No data found',statusCode:1 , "data":0})
        }
        let columns = commonColomns();
            columns.push({ header: 'type', key: 'type', width: 25, style: { font: { size: 12 } } });
            columns.push({ header: 'priority', key: 'priority', width: 25, style: { font: { size: 12 } } });
            columns.push({ header: 'Company Address', key: 'cAddress', width: 25, style: { font: { size: 12 } } });
            columns.push({ header: 'Key Decision Maker Name', key: 'kdm', width: 25, style: { font: { size: 12 } } });
            columns.push({ header: 'Key Decision Maker Phone', key: 'kdmPhone', width: 25, style: { font: { size: 12 } } });
            columns.push({ header: 'Date', key: 'dateTime', width: 25, style: { font: { size: 12 } } });
            columns.push({ header: 'Notes', key: 'notes', width: 25, style: { font: { size: 12 } } });
            for (let i = 0; i < interactions[0].customFields.length; i++) {
                columns.push({ header: interactions[0].customFields[i].dName , key: interactions[0].customFields[i].dName, width: 25, style: { font: { size: 12 } } });
            }
            reportWorkSheet.columns = columns;
            interactions.forEach(interaction => {
                let  assignedTo  = interaction.assigned.toName ? interaction.assigned.toName : '';
                let  assignedFrom  = interaction.assigned.fromName ? interaction.assigned.fromName : '';
                let  companyName = interaction.customer.company ? interaction.customer.company.name : '';
                let  cAddress ;
                let  kdm,kdmPhone ;
                if(interaction.customer.company.address){
                    let companyAddress = interaction.customer.company.address ;           
                    cAddress = companyAddress.street ? companyAddress.street+' ,' : '' ;
                    cAddress = cAddress + (companyAddress.city ? companyAddress.city + ' ,' :'' );
                    cAddress = cAddress + (companyAddress.state ? companyAddress.state +' ,' :'') ;
                    cAddress = cAddress+ (companyAddress.country ? companyAddress.country+' ,':'') ;
                    cAddress = cAddress + (companyAddress.pincode ? companyAddress.pincode +' ,' :'');                    
                }

                if(interaction.customer.company.kdm){
                    kdm = interaction.customer.company.kdm.fName;
                    kdmPhone =  interaction.customer.company.kdm.phone;
                }

                let row = {
                    cName:interaction.customer.fName || '' + ' '+interaction.customer.lName || '',
                    cPhone : interaction.customer.phone.join(','),
                    assignedTName : assignedTo,
                    assignedFName : assignedFrom,
                    cEmail : interaction.email || ' ',
                    source : interaction.source || '',
                    type : interaction.type || '',
                    dateTime : interaction.dateTime ? new Date(interaction.dateTime) : '',
                    notes : interaction.notes || '',
                    priority : interaction.priority === 1 ? "High" : interaction.priority === 2 ? "Medium" : interaction.priority === 3 ? "Low" : " " ,
                    status : interaction.status||'',
                    cCompName : companyName,
                    cAddress : cAddress,
                    kdm :kdm,
                    kdmPhone:kdmPhone
                }

                for (let i = 0; i < interaction.customFields.length; i++) {
                    row[interaction.customFields[i].dName] = interaction.customFields[i].type === "dateTime" ? moment(interaction.customFields[i].value).format('l, h:mm:ss a')  : interaction.customFields[i].value || " ";
                }

                reportWorkSheet.addRow(row);
            });
            return interactions;
    }).then((interactions)=>{
        writeWorkbook(workbook , req);
        res.send({"messgae":"report send sucessfully" , "statusCode":0 ,"data":""})
    }).catch((e)=>{
        console.log(e);
    })
}


function writeWorkbook(workbook , req) {
    workbook.xlsx.writeFile('templates/excel/Report.xlsx')
    .then(function () {
        sendMail(req);
        console.log("report send successfully ");
    });
}

将excel文件写入邮件并发送后。

【问题讨论】：

标签： javascript node.js mongodb exceljs

【解决方案1】：

尝试流式传输：

 // pipe from stream
 const workbook = new Excel.Workbook()
 workbook.useSharedStrings = false

 stream.pipe(workbook.xlsx.createInputStream())

但是，内存管理似乎是这个库的一个持续问题（截至本回答时）。请参阅此 github 问题以供参考：

https://github.com/exceljs/exceljs/issues/709 和这些相关的issues

您可能希望使用另一个库来处理大量 excel 文件（例如：Node-libxl。顺便说一句，此扩展是付费扩展）。

如果你会用Python，你也可以试试OpenPyxl

【讨论】：

如果是这种情况，那么它也不应该发送电子邮件。但目前它正在发送邮件
好的，我试试这个

【解决方案2】：

我用这段代码解决了这个问题。我尝试使用流而不是缓冲区。我使用 Typescript 编写代码，但问题是当数据命中 > 250K 行时，它会说堆内存不足。所以对于大数据，不用Nodejs，用Golang，非常强大，编译写入数据也非常快。

如果您使用它并且仍然遇到堆内存错误，请尝试运行它。

node --max-old-space-size=<enter amount of more memory here> index.js

例如

（6GB-like）：node --max-old-space-size=6124 index.js

或

（12GB-like）：node --max-old-space-size=12028 index.js

而不是像我们通常那样跑步。

node index.js

但不建议设置最大内存，因为它需要更多的 RAM 资源来处理，并且会降低服务器性能。

注意，我使用的是节点 14。

export async function exportToexcel(res: any, jsonSetting: ObjExcelSetting): Promise<any> {

  // Initiate Excel Workbook
  const workBook = new excel.stream.xlsx.WorkbookWriter({
    // the most important part, dont forget to set this
    stream: res
  });

  // initiate into variable from jsonSetting; column, data, sheetname
  let { column, data, sheetname } = jsonSetting;
  // default name for sheet if null
  let defaultsheetname = sheetname ?? "Sheet"
  // Add the worksheet
  const workSheet = workBook.addWorksheet(defaultsheetname);
  // Set the column
  workSheet.columns = column
  // Looping for adding the data to excel
  console.log("Looping for adding the data to excel")
  for (let i = 0; i < data.length; i++) {
    const r = i + 1;
    
    // dont forget to commit for **every** loop
    workSheet.addRow(data[i]).commit();
  }
  // commit the workbook
  console.log("Commit the excel workboom")
  await workBook.commit();
}

res: any -> 指 expressjs 中的响应、请求方法或其他框架（如nestjs）中的类似内容

jsonSetting: ObjExcelSetting -> 引用我的函数、包含、列、工作表名称和 jsondata 的元数据。有关更多信息，请阅读 exceljs 文档，ObjexcelSetting 只是打字稿注释，如果您使用 Javascript，请不要使用它

const workBook = new excel ... -> 指像const excel = require('exceljs')或import * as excel from 'exceljs'这样的启动模块

数据会以Blob的形式返回，所以你猜怎么解析它。

【讨论】：