【问题标题】:Split file to upload with jquery and php: solution拆分文件以使用 jquery 和 php 上传:解决方案
【发布时间】:2020-04-30 11:29:46
【问题描述】:

这是我为绕过网站问题而实施的解决方案。

让我们从问题开始:2 个使用 php 的 apache web 服务器。服务器位于负载平衡器后面。负载均衡器和服务器在 https 中“说话”,但是......如果您尝试上传大于 1MB 的文件,则花费的时间大约为 5 分钟,如果文件为 100KB 或使用直接连接到 Web 服务器,则花费几秒钟。

所以问题出在负载均衡器上:我既不是网络管理员也不是系统管理员,所以我不能说这是流量问题、均衡器上的错误配置还是其他问题。

因此一种解决方法是尝试发送 100KB 的片段(每次上传几秒钟),然后使用 php 脚本在服务器端重新组装所有片段。下一步是使用 jquery 提供的伪并发(提醒 javascript 是单线程的)一次进行更多提交。

怎么做?

我写了完整的解决方案(见下面的答案)只是因为我发现只有不完整的解决方案或引用 wordpress 工具的解决方案(我没有使用)。

注意:由于 memcache,会话在服务器之间共享,因此在 php 端,我们将使用 $_session 变量来确保所有网络服务器上的所有共享数据都可用。

【问题讨论】:

  • 答案必须在不同的帖子中,而不是在问题的帖子中,您可以编辑它以删除答案并将答案作为您问题的答案发布吗?
  • 解决方法在下面的答案中,问题是“如何使用jquery和php实现上传拆分”

标签: javascript php html jquery upload


【解决方案1】:

这就是解决方案。 首先是html代码:

<script src="/js/lib/jquery.min.js" type="text/javascript" ></script><!-- jquery 3.4.1 -->
<form id="fupForm" enctype="multipart/form-data">
    <p id="dbi-upload-progress">Choose a file then click Upload.</p>
    <input id="dbi-file-upload" type="file" name="dbi_import_file" /><br><br>
    <input id="dbi-upload-submit" type="submit" value="Upload" />
</form>

所以,让我们从 javascript 部分开始。这里我们是全局使用的变量:

var max_size   = 5*1024*1024; // Max upload size: 5MB
var slice_size = 100*1024;    // Max size of each fragment: 100 KB
var concurrency = 4;          // Max thread number 
var start_time_total=$.now(); // Time var, just to check upload performance
var file = {};                // Pointer to file to upload
var abort = false;            // Flag used to abort operation
var goals = 0;                // Counter of parallel threads completed
var percent = 0;              // Percent of progress

那么我们需要jquery初始化函数来阻止标准表单提交并启动事务:

$(document).ready(function(e){
    $("#fupForm").on('submit', function(e){
        e.preventDefault(); // prevent standard submit

        // Point to the file
        file = document.querySelector( '#dbi-file-upload' ).files[0];

        // Checking file size and extension
        if(file.size > max_size) {
            alert("Warning! \nFiles bigger than 5MB not supported");
            return;
        } else if(file.name.endsWith('.part')) {
            alert("Warning! \nFiles with .part extension not supported");
            return;
        }

        init_upload_file(); // Beginning the transaction
    });
});

接下来我们需要调用一次我们的 php 脚本来初始化服务器端会话以避免并发和会话共享错误。

function init_upload_file() {
        // Global browser-side variables init, form disable and some console logging
        abort = false; goals = 0; percent = 0; start_time_total=$.now();
        $('#dbi-upload-submit').attr("disabled","disabled");
        $('#dbi-upload-progress').html('Upload init...');
        console.log("Start global transaction @"+ new Date(start_time_total));

        // First call to server with 'init' parameter
        var formData = new FormData();
        formData.append("init", 1);
        var start_time=$.now();
        console.log("start init call @"+new Date(start_time));
        $.ajax( {
            url: "/myPhpFileUrl", type: 'POST', dataType: 'json', contentType: false,
               cache: false, processData:false, data: formData,
            error: function( data ) {
                $('#dbi-upload-progress').html( 'Init call failed' ); // You can use an alert
                console.log( "Init call ko after "+($.now()-start_time)+" msecs; error: "+data );
                console.log( "End global transaction after "+ ($.now()-start_time_total)+" msecs");
                $('#dbi-upload-submit').removeAttr("disabled"); // Re-enable the form submit
            },
            success: function( data ) {
                $( '#dbi-upload-progress' ).html( 'Sending file...' );
                // Calling 'core' function to send multiple fragments
                for(var i=0;i<concurrency;i++)
                    upload_file( (i * (slice_size + 1)) );
            }
        });
}

所以这里是在(js-fake-)多线程模式下调用的“核心”函数:

function upload_file( start ) { // Start argument is the offset used to read a fragment
    if(abort) return; // Check if some thread is in error (abort the transaction)
    // Initialize file reader and put in blob var the fragment to send
    var reader = new FileReader();
    var next_slice = start + slice_size + 1;
    var blob = file.slice( start, next_slice );
    reader.onloadend = function( event ) {
        if ( event.target.readyState !== FileReader.DONE ) { return; }
        // preparing the form data to send and calling the server
        var slice_id=Math.floor(start / slice_size);
        var formData = new FormData();
        formData.append("fileUpload", blob, "." + slice_id + ".part");
        formData.append("part", slice_id);
        var start_time=$.now();
        console.log("Start sending slice "+slice_id+" @"+new Date(start_time));
        $.ajax( {
            url: "/myPhpFileUrl", type: 'POST', dataType: 'json', contentType: false,
            cache: false, processData:false, data: formData,
            error:   function( data ) {
                console.log("slice "+slice_id+" send error after "+($.now()-start_time)+" msecs");
                console.log("error: "+data );
                console.log("End global transaction after "+ ($.now()-start_time_total)+" msecs");
                if(abort) return; // Maybe some other thread was in error...
                // If it's the first error we inform the user and re-enable the form submit
                $('#dbi-upload-progress').html('Upload error'); // You can use an alert
                $('#dbi-upload-submit').removeAttr("disabled");
                abort=true;
            },
            success: function( data ) {
                if(abort) return; // Some other thread is in error, so we exit
                console.log("slice "+slice_id+" ok after "+($.now()-start_time)+" msecs");
                // Calculate progress and next offset
                var size_done = start + slice_size;
                var percent_done = Math.floor( ( size_done / file.size ) * 100 );
                var real_next_slice=start + ((slice_size + 1)* concurrency);
                if ( real_next_slice < file.size ) {
                    // Let's inform the user with the best upload % progress
                    if(percent < percent_done) percent=percent_done;
                    $( '#dbi-upload-progress' ).html( 'Sending file (' + percent + '%)' );
                    upload_file( real_next_slice ); // More to upload, call function recursively
                } else { 
                    // No more fragments to read
                    // if all fragments have been sent we commit the transaction 
                    goals++;
                    if(goals==concurrency) {
                        $('#dbi-upload-progress').html( 'Completing upload...' );
                        commit_uploaded_file();
                    }
                }
            }
        });
    };
    reader.readAsDataURL( blob );
}

现在我们的 jquery 难题的最后一块:提交调用。简而言之,我们最后一次调用我们的 php 脚本来重组文件。

function commit_uploaded_file() {
    var formData = new FormData();
    formData.append("fileUpload", new Blob() , file.name);
    formData.append("fileType", file.type);
    var start_time=$.now();
    console.log("start final commit @"+new Date(start_time));
    $.ajax( {
        url: "/myPhpFileUrl", type: 'POST', dataType: 'json', contentType: false,
        cache: false, processData:false, data: formData,
        error:   function( data ) {
            $( '#dbi-upload-progress' ).html( 'Upload failed' );
            console.log( "Commit transaction ko after "+($.now()-start_time)+" msecs");
            console.log( "Errore: "+data );
            console.log( "End global transaction after "+ ($.now()-start_time_total)+" msecs");
            $('#dbi-upload-submit').removeAttr("disabled");
        },
        success: function( data ) {
            $( '#dbi-upload-progress' ).html( 'Upload completed' );
            console.log( "Commit transaction ok after "+($.now()-start_time)+" msecs");
            console.log( "End global transaction after "+ ($.now()-start_time_total)+" msecs");
            $('#dbi-upload-submit').removeAttr("disabled");
        }
    });
}

就是这样!

...好吧,开个玩笑;-)

我们需要 myPhpFileUrl php 文件:一个 if/else 分支来管理初始化阶段、片段上传阶段和提交阶段。

$uploadedFileId='fileUpload';
$name        = $_FILES[$uploadedFileId]['name'];
$tmp_name    = $_FILES[$uploadedFileId]['tmp_name'];
$type        = $_FILES[$uploadedFileId]['type'];

if($_POST['init'] && $_POST['init']==1)  {
    // Init part, let's clean session and prepare for the upload
    if(isset($_SESSION[session_id()]['UPLOADED_FILE'])) {
        array_splice($_SESSION[session_id()]['UPLOADED_FILE'], 1);
        unset($_SESSION[session_id()]['UPLOADED_FILE']);
    }
    $_SESSION[session_id()]['UPLOADED_FILE'][0]='';
    die(new JsonSuccess("Init session ok")); // Print a simple json string and exit
} else if(substr($name, -5) === '.part') {
    // chunk upload part, this can be executed on different web servers in different threads
    $part=$_POST['part'];
    if(isset($_FILES[$uploadedFileId]['tmp_name']) && is_uploaded_file($_FILES[$uploadedFileId]['tmp_name'])) {
        if($fp = fopen($_FILES[$uploadedFileId]['tmp_name'],'rb')) {
            $contents = '';
            while (!feof($fp)) {
                $contents .= fread($fp, 8192);
            }
            $_SESSION[session_id()]['UPLOADED_FILE'][$part] = $contents;
            die(new JsonSuccess("Chunk ".$name.": loaded ".strlen($contents)." bytes"); // Print a simple json string and exit
        }
    }
} else {
    // Commit part: let's assemble the final file in /tmp folder
    $myfile = fopen("/tmp/".$name, "w")
        or die("Write error: /tmp/".$name);
    for ($row = 0; isset($_SESSION[session_id()]['UPLOADED_FILE'][$row]) ; $row++) {
        fwrite($myfile, $_SESSION[session_id()]['UPLOADED_FILE'][$row]);
    }
    fclose($myfile);

    // Clean the session
    array_splice($_SESSION[session_id()]['UPLOADED_FILE'], 1);
    unset($_SESSION[session_id()]['UPLOADED_FILE']);
    die(new JsonSuccess("Commit success!")); // print a simple json string and exit
}

【讨论】:

    猜你喜欢
    • 2011-09-27
    • 2011-08-20
    • 2014-02-02
    • 2011-08-03
    • 1970-01-01
    • 2019-03-26
    • 1970-01-01
    • 1970-01-01
    • 2013-10-25
    相关资源
    最近更新 更多