scrapy输出csv文件数据多空行问题解决

原文链接：https://blog.csdn.net/hongchen37/article/details/80907743

使用scrapy命令行将数据保存为csv文件时，发现csv多空行。

百度https://stackoverflow.com/questions/39477662/scrapy-csv-file-has-uniform-empty-rows/43394566#43394566

查看源码scrapy.exporters.CsvItemExporter，在io.TextIOWrapper加入参数newline=''，问题就解决啦。

class CsvItemExporter(BaseItemExporter):



 



    def __init__(self, file, include_headers_line=True, join_multivalued=',', **kwargs):




        self._configure(kwargs, dont_fail=True)



        if not self.encoding:



            self.encoding = 'utf-8'




        self.include_headers_line = include_headers_line



        self.stream = io.TextIOWrapper(



            file,



            newline='',




            line_buffering=False,



            write_through=True,



            encoding=self.encoding



        ) if six.PY3 else file



        self.csv_writer = csv.writer(self.stream, **kwargs)



        self._headers_not_written = True




        ...