从前端将客户端文件传递给 webassembly

Question

我希望将用户提交的数据传递给我已编译为 wasm 的 c++ 函数。数据是用户通过输入标签在前端提交的文件，如下所示：

<input type="file" onChange={this.handleFile.bind(this)} />

onChange 回调当前如下所示：

handleFile(e){
    const file = e.currentTarget.files[0];
    const reader = new FileReader();
    reader.onloadend = evt => {
        window.Module.readFile(evt.target.result);
    }
    reader.readAsArrayBuffer(file);
}

最后，包含 readFile 函数的 .cpp 文件如下所示：

void readFile(const std::string & rawString){
  std::vector<uint8_t> data(rawString.begin(), rawString.end());
  //...
}

EMSCRIPTEN_BINDINGS(my_module) {
  emscripten::function("readFile", &readFile);
}

我花了一个下午阅读各种文档，所以我知道我应该在堆上为这些文件分配内存，然后将 ptr 从 js 传递给 readFile，而不是传递所有数据。我的问题是我真的不明白所有这些应该如何工作。谁能解释一下？

Answer 1

借助 Emscripten，您可以为 WASM 使用虚拟文件系统。 首先，您使用-s FORCE_FILESYSTEM=1 选项编译您的C/C++ 代码。 在 C/C++ 中，您只需像往常一样使用标准库函数处理文件。 在 HTML 页面中，您有一个 input type=file 元素。

从输入元素中获取文件并将其传递给 WASM 的示例 JS 代码：

function useFileInput(fileInput) {
    if (fileInput.files.length == 0)
        return;
    var file = fileInput.files[0];

    var fr = new FileReader();
    fr.onload = function () {
        var data = new Uint8Array(fr.result);

        Module['FS_createDataFile']('/', 'filename', data, true, true, true);
        Module.ccall('YourCppFunctionToUtilizeTheFile', null, [], null);

        fileInput.value = '';
    };
    fr.readAsArrayBuffer(file);
}

链接：

1. Emscripten - File System Overview
2. Here I use the approach, see emulatorAttachFileInput() function

Answer 2

这是部分答案。它比我最初做的更好，我觉得它可能更接近创作者的意图。但是，我仍在创建该文件的多个副本。感谢this post for making it click for me.

现在这是我的 handleFile 回调，用我学到的东西进行了评论。

handleFile(e){

    const file = e.currentTarget.files[0];
    if(!(file instanceof Blob)) return;
    const reader = new FileReader();
    reader.onloadend = evt => {

        //evt.target.result is an ArrayBuffer. In js, 
        //you can't do anything with an ArrayBuffer 
        //so we have to ???cast??? it to an Uint8Array
        const uint8_t_arr = new Uint8Array(evt.target.result);

        //Right now, we have the file as a unit8array in javascript memory. 
        //As far as I understand, wasm can't directly access javascript memory. 
        //Which is why we need to allocate special wasm memory and then
        //copy the file from javascript memory into wasm memory so our wasm functions 
        //can work on it.

        //First we need to allocate the wasm memory. 
        //_malloc returns the address of the new wasm memory as int32.
        //This call is probably similar to 
        //uint8_t * ptr = new uint8_t[sizeof(uint8_t_arr)/sizeof(uint8_t_arr[0])]
        const uint8_t_ptr = window.Module._malloc(uint8_t_arr.length);

        //Now that we have a block of memory we can copy the file data into that block
        //This is probably similar to 
        //std::memcpy(uint8_t_ptr, uint8_t_arr, sizeof(uint8_t_arr)/sizeof(uint8_t_arr[0]))
        window.Module.HEAPU8.set(uint8_t_arr, uint8_t_ptr);

        //The only thing that's now left to do is pass 
        //the address of the wasm memory we just allocated
        //to our function as well as the size of our memory.
        window.Module.readFile(uint8_t_ptr, uint8_t_arr.length);

        //At this point we're forced to wait until wasm is done with the memory. 
        //Your site will now freeze if the memory you're working on is big. 
        //Maybe we can somehow let our wasm function run on a seperate thread and pass a callback?

        //Retreiving our (modified) memory is also straight forward. 
        //First we get some javascript memory and then we copy the 
        //relevant chunk of the wasm memory into our javascript object.
        const returnArr = new Uint8Array(uint8_t_arr.length);
        //If returnArr is std::vector<uint8_t>, then is probably similar to 
        //returnArr.assign(ptr, ptr + dataSize)
        returnArr.set(window.Module.HEAPU8.subarray(uint8_t_ptr, uint8_t_ptr + uint8_t_arr.length));

        //Lastly, according to the docs, we should call ._free here.
        //Do we need to call the gc somehow?
        window.Module._free(uint8_t_ptr);

    }
    reader.readAsArrayBuffer(file);
}

这里是 readFile.cpp。

#include <emscripten/bind.h>

//We get out pointer as a plain int from javascript
void readFile(const int & addr, const size_t & len){
  //We use a reinterpret_cast to turn our plain int into a uint8_t pointer. After
  //which we can play with the data just like we would normally.
  uint8_t * data = reinterpret_cast<uint8_t *>(addr);
  for(size_t i = 0; i < len; ++i){
    data[i] += 1;
  }
}

//Using this command to compile
//  emcc --bind -O3 readFile.cpp -s WASM=1 -s TOTAL_MEMORY=268435456 -o api.js --std=c++11
//Note that you need to make sure that there's enough memory available to begin with.
//I got only 16mb without passing the TOTAL_MEMORY setting.
EMSCRIPTEN_BINDINGS(my_module) {
  emscripten::function("readFile", &readFile);
}