从 URL 复制的文件被截断

Question

我在复制文件时遇到问题。我的代码：

$file = "https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml";

$newfile = $_SERVER['DOCUMENT_ROOT'] . '/input/PO_Offerte_G_MLIBERO_20190130.xml';

if(copy($file, $newfile)) {
    echo "salvato<br>";
} else {
    echo "ERROR inport file PO_Offerte_".$data.".".$ext."<br>";
    die;
}

copy() 为真，文件已创建，但文件末尾的某些行丢失...文件为 3.6MB，文件末尾的 0.3 丢失...

如果我手动下载文件一切都很好，所以源是完整的......

如果我使用file_get_contents() 获取文件内容并尝试使用文件写入功能将其保存在文件中，我实际上会遇到同样的问题...

我不认为upload_max_filesize 和post_max_size 实际上涉及copy()，但它们设置为20MB

有什么建议吗？

谢谢

Answer 1

由于 URL 启用了 SSL，因此通常需要打包附加信息并随请求一起发送 - 对于 copy 函数，有一个 context 参数。 context 允许您指定方法、协议等来支持请求。

$url='https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml';

/* download a copy from: https://curl.haxx.se/ca/cacert.pem */
$cacert=__DIR__ . DIRECTORY_SEPARATOR . 'cacert.pem';
/* define where files are to be stored */
$dir='c:/temp/downloads/';

使用stream_context_create() 函数为请求构建上下文，并使用copy 实际发出请求。

$filepath=$dir . basename( $url );

$args = array(
    'http'  =>  array( 'method' => 'GET', 'protocol_version' => '1.1' ),
    'ssl'   =>  array( 'verify_peer' => true, 'verify_peer_name' => true, 'allow_self_signed' => false, 'cafile' => $cacert )
);
$ctxt = stream_context_create( $args );
$status = copy( $url, $filepath, $ctxt );

if( $status && file_exists( $filepath ) ){
    printf( 
        'The file "%s" downloaded successfully. %sMb written to disk.',
        $filepath,
        round( filesize( $filepath ) / pow( 1024, 2 ),2 )
    );
}

另一个也许更好的选择是 curl：

function downloadfile( $url=false, $dir=false, $cacert=false ){
    if( $url && $dir ){

        /* define the save path */
        $filepath = $dir . basename( $url );

        /* time how long the download takes */
        $start=time();

        /* open a file pointer for use by curl */
        $fp = fopen( $filepath, 'w+' );

        /* create the curl request - write file directly */
        $ch = curl_init( $url );
        curl_setopt($ch, CURLOPT_HEADER, 0 );
        curl_setopt($ch, CURLOPT_TIMEOUT, 10 );
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );
        curl_setopt($ch, CURLOPT_BINARYTRANSFER, true );
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true );
        curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36' );
        if( parse_url( $url, PHP_URL_SCHEME )=='https' ){
            curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true );
            curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2 );
            curl_setopt($ch, CURLOPT_CAINFO, $cacert );
        }
        curl_setopt($ch, CURLOPT_ENCODING, '' );
        curl_setopt($ch, CURLOPT_FILE, $fp );# write to file

        /* the response */
        $obj=(object)array(
            'response'  =>  curl_exec($ch),
            'info'      =>  (object)curl_getinfo($ch),
            'error'     =>  curl_error($ch),
            'filepath'  =>  $filepath
        );

        /* tidy up */
        curl_close($ch);
        fclose($fp);

        /* calculate time operation took */
        $obj->duration=round( time() - $start, 2 );

        return $obj;
    }
}



/* run the function */
$obj = downloadfile( $url, $dir, $cacert );

if( $obj->info->http_code==200 ){
    printf( 
        'The file "%s" downloaded successfully in approximately %ss. %sMb written to disk.',
        $obj->filepath,
        $obj->duration,
        round( filesize( $obj->filepath ) / pow( 1024, 2 ),2 )
    );
} else {
    printf(
        'Error: A problem was encountered downloading %s. The response code is: %d and error message: "%s"',
        $url,
        $obj->info->http_code,
        $obj->error
    );
}

以上两种方法和手动下载的结果是3.19Mb的文件

Answer 2

问题出在 Http 1.0

这应该可以解决它：

$sc = stream_context_create(['http' => ['protocol_version' => '1.1']]);

copy($file, $newfile, $sc);

Answer 3

我能够使用file_get_contents 并强制使用 HTTP/1.1 协议：

$context = stream_context_create([
    'http' => [
      'protocol_version' => '1.1',
      'header' => 'Connection: Close'
    ],
]);
$content = file_get_contents('https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml', false, $context);
file_put_contents('document.xml', $content);

话虽如此，我还是推荐使用 CURL：

$ch = curl_init();
$curlopts = array();
$curlopts[CURLOPT_RETURNTRANSFER] = true;
$curlopts[CURLOPT_VERBOSE] = true;
$curlopts[CURLOPT_URL] = 'https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml';
curl_setopt_array($ch, $curlopts);
$content = curl_exec($ch);
file_put_contents('document.xml', $content);
curl_close($ch);

Answer 4

看起来像 memory_limit。尝试在脚本开头将其设置为更高的值。

ini_set('memory_limit' '1024m')

Answer 5

http://php.net/manual/en/function.copy.php

我看到了这个：exec("xcopy $source $destination");

试试看！

Answer 6

请试试这个连接超时功能，希望这对你有用。

$url=curl_init("https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml"); 
$set_timeout=400; 
$newfile=$_SERVER['DOCUMENT_ROOT'] . '/input/PO_Offerte_G_MLIBERO_20190130.xml';
curl_setopt($url, CURLOPT_CONNECTTIMEOUT, $set_timeout); 
$content = curl_exec($url);//execute request
if($content)
{
    $copied_file = fopen($newfile, "w"); 
    if(fwrite($copied_file , $content)){
        echo "Done successfully";
    }
    else{
        echo "unable to write file";
        fclose($copied_file ); 
    }
}
else {
    echo "Something Went wrong";
}

Answer 7

您遇到了问题，因为您使用的是 HTTPS 协议。 copy() 和 file_get_content() 函数在使用 HTTPS 时存在一些问题。 加载远程文件更可靠的方法是使用 CURL。例如：

$url = 'https://www.ilportaleofferte.it/portaleOfferte/resources/opendata/csv/offerteML/2019_1/PO_Offerte_G_MLIBERO_20190130.xml';    
$filePath = $_SERVER['DOCUMENT_ROOT'] . '/input/PO_Offerte_G_MLIBERO_20190130.xml';

$ch = curl_init($url);
$fp = fopen($filePath, 'wb');
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
fclose($fp);

Answer 8

看看你的 php.ini。 可能你必须增加这两个参数

upload_max_filesize 和 post_max_size